Attention: to Better Stand on the Shoulders of Giants

TL;DR

This paper introduces an attention mechanism for predicting long-term scientific impact using large-scale citation data, revealing that focusing attention enhances impact prediction and challenges traditional assumptions about citation distributions.

Contribution

It proposes a novel attention-based model for impact prediction and demonstrates that emphasizing limited attention improves long-term impact forecasting.

Findings

Attention mechanism outperforms traditional models in impact prediction

Limited attention focus better captures influence of foundational works

Results challenge conventional power-law assumptions in citation distributions

Abstract

Science of science (SciSci) is an emerging discipline wherein science is used to study the structure and evolution of science itself using large data sets. The increasing availability of digital data on scholarly outcomes offers unprecedented opportunities to explore SciSci. In the progress of science, the previously discovered knowledge principally inspires new scientific ideas, and citation is a reasonably good reflection of this cumulative nature of scientific research. The researches that choose potentially influential references will have a lead over the emerging publications. Although the peer review process is the mainly reliable way of predicting a paper's future impact, the ability to foresee the lasting impact based on citation records is increasingly essential in the scientific impact analysis in the era of big data. This paper develops an attention mechanism for the long-term scientific impact prediction and validates the method based on a real large-scale citation data set. The results break conventional thinking. Instead of accurately simulating the original power-law distribution, emphasizing the limited attention can better stand on the shoulders of giants.