A Qualitative Evaluation of User Preference for Link-based vs. Text-based Recommendations of Wikipedia Articles

TL;DR

This study qualitatively compares user perceptions of link-based and text-based Wikipedia article recommendations, revealing strengths and weaknesses of each approach and suggesting hybrid systems for improved user satisfaction.

Contribution

It provides qualitative insights into user preferences for link- and text-based recommendations, highlighting the potential of hybrid approaches tailored to user needs.

Findings

Text-based recommendations scored higher on similarity satisfaction.

Link-based recommendations scored higher on diversity and serendipity.

Hybrid approaches can better meet diverse user preferences.

Abstract

Literature recommendation systems (LRS) assist readers in the discovery of relevant content from the overwhelming amount of literature available. Despite the widespread adoption of LRS, there is a lack of research on the user-perceived recommendation characteristics for fundamentally different approaches to content-based literature recommendation. To complement existing quantitative studies on literature recommendation, we present qualitative study results that report on users' perceptions for two contrasting recommendation classes: (1) link-based recommendation represented by the Co-Citation Proximity (CPA) approach, and (2) text-based recommendation represented by Lucene's MoreLikeThis (MLT) algorithm. The empirical data analyzed in our study with twenty users and a diverse set of 40 Wikipedia articles indicate a noticeable difference between text- and link-based recommendation generation approaches along several key dimensions. The text-based MLT method receives higher satisfaction ratings in terms of user-perceived similarity of recommended articles. In contrast, the CPA approach receives higher satisfaction scores in terms of diversity and serendipity of recommendations. We conclude that users of literature recommendation systems can benefit most from hybrid approaches that combine both link- and text-based approaches, where the user's information needs and preferences should control the weighting for the approaches used. The optimal weighting of multiple approaches used in a hybrid recommendation system is highly dependent on a user's shifting needs.