Performing Informetric Analysis on Information Retrieval Test Collections: Preliminary Experiments in the Physics Domain

TL;DR

This paper explores the integration of informetric analysis with information retrieval evaluation using the iSearch collection from arXiv.org, aiming to enhance retrieval and recommendation systems in the physics domain.

Contribution

It demonstrates the feasibility of using the iSearch test collection for evaluating informetric methods in information retrieval, linking bibliometric analysis with retrieval performance.

Findings

Identified key authors relevant to physics research questions

Confirmed the presence of important authors and documents in the corpus

Established a preliminary connection between informetric analysis and retrieval evaluation

Abstract

The combination of informetric analysis and information retrieval allows a twofold application. (1) While in-formetrics analysis is primarily used to gain insights into a scientific domain, it can be used to build recommen-dation or alternative ranking services. They are usually based on methods like co-occurrence or citation analyses. (2) Information retrieval and its decades-long tradition of rigorous evaluation using standard document corpora, predefined topics and relevance judgements can be used as a test bed for informetric analyses. We show a preliminary experiment on how both domains can be connected using the iSearch test collection, a standard information retrieval test collection derived from the open access arXiv.org preprint server. In this paper the aim is to draw a conclusion about the appropriateness of iSearch as a test bed for the evaluation of a retrieval or recommendation system that applies informetric methods to improve retrieval results for the user. Based on an interview study with physicists, bibliographic coupling and author-co-citation analysis, important authors for ten different research questions are identified. The results show that the analysed corpus includes these authors and their corresponding documents. This study is a first step towards a combination of retrieval evaluations and the evaluation of informetric analyses methods.