I'm doing research using the WiseWire system
(http://www.wisewire.com), which uses both adaptive
content-based filtering and adaptive collaborative
filtering to personalize the user's view of the web.
I've got a huge corpus of both human-rated and unlabelled
documents, some interested new approaches to the problem
and some evaluation metrics that are unique for this
domain.
What I don't have is strong comparisons with other work
in the area. Can anyone suggest good benchmarks that can
be used to for comparison purposes? We've made some
stabs at it, but we'd like to do better and then publish
the results of the comparison. The benchmarks we're
looking for are:
1) corpora/data sets
2) algorithms
3) evaluation metrics
4) results of other similar work
Ultimately we'd like to come up with an industry-standard
way for these kinds of systems to be measured in general
for everyone to use.
Thanks for any light the collective wisdom here can shed,
-- Ken Lang