The Operationalization of "Fields" as WoS Subject Categories (WCs) in Evaluative Bibliometrics: The cases of "Library and Information Science" and "Science & Technology Studies"

TL;DR

This paper critically examines the use of Web-of-Science Subject Categories for citation normalization, revealing limitations due to indexer effects and suggesting the need for improved practices in evaluative bibliometrics.

Contribution

It demonstrates that current WC classifications lack analytical clarity for bibliometric normalization and advocates for developing better classification practices.

Findings

WCs are based on outdated, machine-based classifications with manual corrections.

Indexing effects distort bibliometric normalization.

Current practices may not be optimal for evaluation purposes.

Abstract

Normalization of citation scores using reference sets based on Web-of-Science Subject Categories (WCs) has become an established ("best") practice in evaluative bibliometrics. For example, the Times Higher Education World University Rankings are, among other things, based on this operationalization. However, WCs were developed decades ago for the purpose of information retrieval and evolved incrementally with the database; the classification is machine-based and partially manually corrected. Using the WC "information science & library science" and the WCs attributed to journals in the field of "science and technology studies," we show that WCs do not provide sufficient analytical clarity to carry bibliometric normalization in evaluation practices because of "indexer effects." Can the compliance with "best practices" be replaced with an ambition to develop "best possible practices"? New research questions can then be envisaged.