Evaluating Recommendation Algorithms by Graph Analysis

TL;DR

This paper introduces a graph-based framework for evaluating recommendation algorithms by analyzing the connectivity and reachability within the underlying data structure, providing insights beyond traditional accuracy metrics.

Contribution

It proposes a novel graph analysis framework to evaluate recommendation algorithms based on reachability and connectivity, complementing existing accuracy-based assessments.

Findings

Analyzed the impact of algorithm parameters on graph properties

Demonstrated the framework using movie recommender datasets

Provided insights into dataset connectivity and path lengths

Abstract

We present a novel framework for evaluating recommendation algorithms in terms of the `jumps' that they make to connect people to artifacts. This approach emphasizes reachability via an algorithm within the implicit graph structure underlying a recommender dataset, and serves as a complement to evaluation in terms of predictive accuracy. The framework allows us to consider questions relating algorithmic parameters to properties of the datasets. For instance, given a particular algorithm `jump,' what is the average path length from a person to an artifact? Or, what choices of minimum ratings and jumps maintain a connected graph? We illustrate the approach with a common jump called the `hammock' using movie recommender datasets.