An evaluation of Bradfordizing effects

TL;DR

This paper evaluates Bradfordizing, a bibliometric method, for improving information retrieval by re-ranking core scientific documents across multiple disciplines, demonstrating significant relevance improvements in digital libraries.

Contribution

It provides a systematic evaluation of Bradfordizing's effectiveness in re-ranking core documents in diverse scientific fields using controlled experiments.

Findings

Bradfordizing effectively re-ranks core documents in digital libraries.

Relevance of core documents is significantly higher after Bradfordizing.

Statistical tests confirm the significance of relevance improvements.

Abstract

The purpose of this paper is to apply and evaluate the bibliometric method Bradfordizing for information retrieval (IR) experiments. Bradfordizing is used for generating core document sets for subject-specific questions and to reorder result sets from distributed searches. The method will be applied and tested in a controlled scenario of scientific literature databases from social and political sciences, economics, psychology and medical science (SOLIS, SoLit, USB Koeln Opac, CSA Sociological Abstracts, World Affairs Online, Psyndex and Medline) and 164 standardized topics. An evaluation of the method and its effects is carried out in two laboratory-based information retrieval experiments (CLEF and KoMoHe) using a controlled document corpus and human relevance assessments. The results show that Bradfordizing is a very robust method for re-ranking the main document types (journal articles and monographs) in today's digital libraries (DL). The IR tests show that relevance distributions after re-ranking improve at a significant level if articles in the core are compared with articles in the succeeding zones. The items in the core are significantly more often assessed as relevant, than items in zone 2 (z2) or zone 3 (z3). The improvements between the zones are statistically significant based on the Wilcoxon signed-rank test and the paired T-Test.