Are topic-specific search term, journal name and author name recommendations relevant for researchers?

TL;DR

This study evaluates the relevance of bibliometric-enhanced IR recommendations for search terms, journal names, and author names in social science research, showing high relevance and potential for integration into digital library searches.

Contribution

It provides empirical evidence that bibliometric-enhanced IR recommendations are highly relevant for researchers and can improve search effectiveness in digital libraries.

Findings

Average precision for top recommendations: 0.75 for authors, 0.74 for search terms, 0.73 for journals.

Relevance varies across topics and researcher types.

Practitioners favor author recommendations; postdocs prefer journal recommendations.

Abstract

In this paper we describe a case study where researchers in the social sciences (n=19) assess topical relevance for controlled search terms, journal names and author names which have been compiled automatically by bibliometric-enhanced information retrieval (IR) services. We call these bibliometric-enhanced IR services Search Term Recommender (STR), Journal Name Recommender (JNR) and Author Name Recommender (ANR) in this paper. The researchers in our study (practitioners, PhD students and postdocs) were asked to assess the top n pre-processed recommendations from each recommender for specific research topics which have been named by them in an interview before the experiment. Our results show clearly that the presented search term, journal name and author name recommendations are highly relevant to the researchers' topic and can easily be integrated for search in Digital Libraries. The average precision for top ranked recommendations is 0.75 for author names, 0.74 for search terms and 0.73 for journal names. The relevance distribution differs largely across topics and researcher types. Practitioners seem to favor author name recommendations while postdocs have rated author name recommendations the lowest. In the experiment the small postdoc group (n=3) favor journal name recommendations.