When a Paper Has 1000 Authors: Rethinking Citation Metrics in the Era of LLMs

TL;DR

This paper introduces the SBCI index, a new citation metric designed to better assess individual researcher impact in large-scale collaborations involving thousands of authors, addressing limitations of traditional metrics.

Contribution

The paper proposes the SBCI index, analyzes its theoretical properties, and evaluates its effectiveness on synthetic datasets, improving impact assessment in large-scale research collaborations.

Findings

SBCI offers a more discriminative impact measure.

Traditional metrics like h-index are inadequate for large collaborations.

SBCI demonstrates robustness in synthetic data evaluations.

Abstract

Author-level citation metrics provide a practical, interpretable, and scalable signal of scholarly influence in a complex research ecosystem. It has been widely used as a proxy in hiring decisions. However, the past five years have seen the rapid emergence of large-scale publications in the field of large language models and foundation models, with papers featuring hundreds to thousands of co-authors and receiving tens of thousands of citations within months. For example, Gemini has 1361 authors and has been cited around 4600 times in 19 months. In such cases, traditional metrics, such as total citation count and the $h$-index, fail to meaningfully distinguish individual contributions. Therefore, we propose the following research question: How can one identify standout researchers among thousands of co-authors in large-scale LLM papers? This question is particularly important in scenarios such as academic hiring and funding decisions. In this paper, we introduce a novel citation metric designed to address this challenge by balancing contributions across large-scale and small-scale publications. We propose the SBCI index, analyze its theoretical properties, and evaluate its behavior on synthetic publication datasets. Our results demonstrate that the proposed metric provides a more robust and discriminative assessment of individual scholarly impact in the era of large-scale collaborations.