Evaluating collaborative filtering recommender systems

If you are interested in Recommender Systems, you cannot miss Evaluating collaborative filtering recommender systems by Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen and John T. Riedl.
You need an ACM Web Account and some time (it is 53 pages).

Recommender systems have been evaluated in many, often incomparable, ways. In this article, we review the key decisions in evaluating collaborative filtering recommender systems: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole. In addition to reviewing the evaluation strategies used by prior researchers, we present empirical results from the analysis of various accuracy metrics on one content domain where all the tested metrics collapsed roughly into three equivalence classes. Metrics within each equivalency class were strongly correlated, while metrics from different equivalency classes were uncorrelated.

Subscribe to RSS Feed If you enjoyed reading this, subscribe to my RSS Feed
(you can always unsubscribe later)

14 Responses to this post.

  1. Cai's Gravatar

    Posted by Cai on 09.04.04 at 5:15 pm

    The article is really excellent and represents the current state-of-the-art in research coping with RS evaluation frameworks. The entire ACM TOIS special issue on RS (Jan 2004), where Herlocker’s work is part of, is excellent and recommended to all those into that subject matter …

  2. Web Dawn - Rebirth of the Social Marketplace's Gravatar

    Posted by Web Dawn - Rebirth of the Social Marketplace on 09.04.04 at 5:15 pm

    Trust, Recommender Systems, and Social Software

    That’s what is says at the top of Paolo Massa’s blog: “Ramblings on Trust, Recommender Systems, Social Software, and much more.” Wow, isn’t that a great combination? Personally, I believe that the combination of these things has the potential to become…

  3. Web Dawn - Rebirth of the Social Marketplace's Gravatar

    Posted by Web Dawn - Rebirth of the Social Marketplace on 09.04.04 at 5:15 pm

    Trust, Recommender Systems, and Social Software

    That’s what is says at the top of Paolo Massa’s blog: “Ramblings on Trust, Recommender Systems, Social Software, and much more.” Wow, isn’t that a great combination? Personally, I believe that the combination of these things has the potential to become…

  4. Web Dawn - Rebirth of the Social Marketplace's Gravatar

    Posted by Web Dawn - Rebirth of the Social Marketplace on 09.04.04 at 5:15 pm

    Trust, Recommender Systems, and Social Software

    That’s what is says at the top of Paolo Massa’s blog: “Ramblings on Trust, Recommender Systems, Social Software, and much more.” Wow, isn’t that a great combination? Personally, I believe that the combination of these things has the potential to become…

  5. Web Dawn - Rebirth of the Social Marketplace's Gravatar

    Posted by Web Dawn - Rebirth of the Social Marketplace on 09.04.04 at 5:15 pm

    Trust, Recommender Systems, and Social Software

    That’s what is says at the top of Paolo Massa’s blog: “Ramblings on Trust, Recommender Systems, Social Software, and much more.” Wow, isn’t that a great combination? Personally, I believe that the combination of these things has the potential to become…

  6. Web Dawn - Rebirth of the Social Marketplace's Gravatar

    Posted by Web Dawn - Rebirth of the Social Marketplace on 09.04.04 at 5:15 pm

    Trust, Recommender Systems, and Social Software

    That’s what is says at the top of Paolo Massa’s blog: “Ramblings on Trust, Recommender Systems, Social Software, and much more.” Wow, isn’t that a great combination? Personally, I believe that the combination of these things has the potential to become…

  7. Web Dawn - Rebirth of the Social Marketplace's Gravatar

    Posted by Web Dawn - Rebirth of the Social Marketplace on 09.04.04 at 5:15 pm

    Trust, Recommender Systems, and Social Software

    That’s what is says at the top of Paolo Massa’s blog: “Ramblings on Trust, Recommender Systems, Social Software, and much more.” Wow, isn’t that a great combination? Personally, I believe that the combination of these things has the potential to become…

  8. crap cleaner an's Gravatar

    Posted by crap cleaner an on 09.04.04 at 5:15 pm

  9. crap cleaner an's Gravatar

    Posted by crap cleaner an on 09.04.04 at 5:15 pm

  10. crap cleaner an's Gravatar

    Posted by crap cleaner an on 09.04.04 at 5:15 pm

  11. crap cleaner an's Gravatar

    Posted by crap cleaner an on 09.04.04 at 5:15 pm

  12. crap cleaner an's Gravatar

    Posted by crap cleaner an on 09.04.04 at 5:15 pm

  13. Enily Drury's Gravatar

    Posted by Enily Drury on 09.04.04 at 5:15 pm

    grade 8

    Evaluating collaborati…

  14. Enily Drury's Gravatar

    Posted by Enily Drury on 09.04.04 at 5:15 pm

    grade 8

    Evaluating collaborati…

Respond to this post