Exploring user needs in relation to algorithmically constructed classifications of publications

TL;DR

This study evaluates the practical usefulness of algorithmically generated publication classifications for researchers and managers in a university setting, highlighting their role in understanding research areas and supporting decision-making.

Contribution

It provides qualitative insights into how classification maps support research management and decision-making in an academic context, an area previously underexplored.

Findings

Cluster maps aid in understanding research areas

Interactivity improves interpretation of classifications

Qualitative evaluation complements quantitative assessments

Abstract

Algorithmic classification of research publications has been created to study different aspects of research. Such classifications can be used to support information needs in universities for decision making. However, the classifications have foremost been evaluated quantitatively regarding their content, but not qualitatively regarding their feasibility in a specific context. The aim of this study was to explore and evaluate the usefulness of such classifications to users in the context of exploring an emerging research area. I conducted four interviews with managers of a project aimed to support research and application of artificial intelligence at the Swedish medical university Karolinska Institutet. The interviews focused on the information need of the managers. To support the project, a classification was created by clustering of publications in a citation network. A cluster map based on this classification was provided to the project leader and one interview focused on the use of the classification in the project in relation to the stated information needs. The interviews showed that the aim of the project was to improve competence, enhance communication between researchers and develop support structures. Getting an overview of artificial intelligence at the university and information about who is doing what was important to fulfill this aim. The cluster map was used to support activities conducted by the project leader, such as interviews and information gathering. It was also used to get overview and display of AI research at KI. Interpretation was found to be challenging in some cases. The interactivity of the map facilitated interpretation. This study was small in scope, but it provides one piece of knowledge about the information needs related to algorithmic classifications.