Real World Evaluation of Approaches to Research Paper Recommendation

TL;DR

This paper evaluates various research paper recommendation methods using real user click data, finding that term-based similarity search outperforms keyword-based approaches, providing insights for future improvements.

Contribution

It establishes a set of baseline approaches for research paper recommendation and evaluates them on real-world data, highlighting the effectiveness of term-based similarity search.

Findings

Term-based similarity search outperforms keyword-based methods

Evaluation conducted on real user click data

Provides a foundation for future performance improvements

Abstract

In this work, we have identified the need for choosing baseline approaches for research-paper recommendation systems. Following a literature survey of all research paper recommendation approaches described over the last four years, we framed criteria that makes for a well-rounded set of baselines. These are implemented on Mr. DLib a literature recommendation platform. User click data was collected as part of an ongoing experiment in collaboration with our partner Gesis. We reported the results from our evaluation for the experiments. We will be able to draw clearer conclusions as time passes. We find that a term based similarity search performs better than keyword based approaches. These results are a good starting point in finding performance improvements for related document searches.