Algorithmically generated subject categories based on citation relations: An empirical micro study using papers on overall water splitting

TL;DR

This study evaluates an algorithmic classification system based on citation relations for water splitting research, comparing it with traditional classifications and analyzing its effectiveness in identifying relevant papers and citation impact.

Contribution

The paper provides an empirical assessment of the ACCS, a citation-based classification system, and compares its performance with traditional intellectual classifications.

Findings

ACCS shows limited discriminatory power for water splitting papers.

Traditional classifications outperform ACCS in identifying relevant papers.

Citation impact varies across ACCS clusters and traditional categories.

Abstract

One important reason for the use of field categorization in bibliometrics is the necessity to make citation impact of papers published in different scientific fields comparable with each other. Raw citations are normalized by using field-normalization schemes to achieve comparable citation scores. There are different approaches to field categorization available. They can be broadly classified as intellectual and algorithmic approaches. A paper-based algorithmically constructed classification system (ACCS) was proposed which is based on citation relations. Using a few ACCS field-specific clusters, we investigate the discriminatory power of the ACCS. The micro study focusses on the topic "overall water splitting" and related topics. The first part of the study investigates intellectually whether the ACCS is able to identify papers on overall water splitting reliably and validly. Next, we compare the ACCS with (1) a paper-based intellectual (INSPEC) classification and (2) a journal-based intellectual classification (Web of Science, WoS, subject categories). In the last part of our case study, we compare the average number of citations in selected ACCS clusters (on overall water splitting and related topics) with the average citation count of publications in WoS subject categories related to these clusters. The results of this micro study question the discriminatory power of the ACCS. We recommend larger follow-up studies on broad datasets.