Citations: Indicators of Quality? The Impact Fallacy

TL;DR

This paper argues that citations serve dual roles: short-term as currency at the research front and long-term as indicators of knowledge codification, questioning their reliability as quality measures.

Contribution

It introduces a nuanced view of citations, distinguishing their short-term and long-term functions, and demonstrates this with empirical case studies and the Multi-RPYS method.

Findings

Short-term citations reflect research front involvement.

Long-term citations indicate knowledge codification.

Multi-RPYS distinguishes impact types.

Abstract

We argue that citation is a composed indicator: short-term citations can be considered as currency at the research front, whereas long-term citations can contribute to the codification of knowledge claims into concept symbols. Knowledge claims at the research front are more likely to be transitory and are therefore problematic as indicators of quality. Citation impact studies focus on short-term citation, and therefore tend to measure not epistemic quality, but involvement in current discourses in which contributions are positioned by referencing. We explore this argument using three case studies: (1) citations of the journal Soziale Welt as an example of a venue that tends not to publish papers at a research front, unlike, for example, JACS; (2) Robert Merton as a concept symbol across theories of citation; and (3) the Multi-RPYS ("Multi-Referenced Publication Year Spectroscopy") of the journals Scientometrics, Gene, and Soziale Welt. We show empirically that the measurement of "quality" in terms of citations can further be qualified: short-term citation currency at the research front can be distinguished from longer-term processes of incorporation and codification of knowledge claims into bodies of knowledge. The recently introduced Multi-RPYS can be used to distinguish between short-term and long-term impacts.