Citations versus expert opinions: Citation analysis of Featured Reviews of the American Mathematical Society

TL;DR

This study compares peer-reviewed featured reviews and citation counts in mathematics, revealing they identify largely different sets of significant articles, highlighting their independent roles in research evaluation.

Contribution

It provides an empirical analysis of the relationship between peer review and citation metrics in mathematics, showing their independence in identifying important research.

Findings

Featured reviews and highly cited articles largely do not overlap.

Peer review and citation metrics offer different perspectives on research significance.

Mathematics serves as a controlled environment to study these evaluation methods.

Abstract

Peer review and citation metrics are two means of gauging the value of scientific research, but the lack of publicly available peer review data makes the comparison of these methods difficult. Mathematics can serve as a useful laboratory for considering these questions because as an exact science, there is a narrow range of reasons for citations. In mathematics, virtually all published articles are post-publication reviewed by mathematicians in Mathematical Reviews (MathSciNet) and so the data set was essentially the Web of Science mathematics publications from 1993 to 2004. For a decade, especially important articles were singled out in Mathematical Reviews for featured reviews. In this study, we analyze the bibliometrics of elite articles selected by peer review and by citation count. We conclude that the two notions of significance described by being a featured review article and being highly cited are distinct. This indicates that peer review and citation counts give largely independent determinations of highly distinguished articles. We also consider whether hiring patterns of subfields and mathematicians' interest in subfields reflect subfields of featured review or highly cited articles. We reexamine data from two earlier studies in light of our methods for implications on the peer review/citation count relationship to a diversity of disciplines.