How many preprints have actually been printed and why: a case study of computer science preprints on arXiv

TL;DR

This study analyzes the publication rate of computer science preprints on arXiv from 2008 to 2017, introduces a BERT-based mapping method to link preprints with final publications, and identifies factors influencing publication success.

Contribution

It presents a novel semantics-based BERT method for accurately matching preprints to published papers and provides insights into the characteristics of preprints that are eventually published.

Findings

66% of preprints are published with unchanged titles

11% are published under different titles with modifications

Published preprints have more revisions, authors, detailed abstracts, references, and source code

Abstract

Preprints play an increasingly critical role in academic communities. There are many reasons driving researchers to post their manuscripts to preprint servers before formal submission to journals or conferences, but the use of preprints has also sparked considerable controversy, especially surrounding the claim of priority. In this paper, a case study of computer science preprints submitted to arXiv from 2008 to 2017 is conducted to quantify how many preprints have eventually been printed in peer-reviewed venues. Among those published manuscripts, some are published under different titles and without an update to their preprints on arXiv. In the case of these manuscripts, the traditional fuzzy matching method is incapable of mapping the preprint to the final published version. In view of this issue, we introduce a semantics-based mapping method with the employment of Bidirectional Encoder Representations from Transformers (BERT). With this new mapping method and a plurality of data sources, we find that 66% of all sampled preprints are published under unchanged titles and 11% are published under different titles and with other modifications. A further analysis was then performed to investigate why these preprints but not others were accepted for publication. Our comparison reveals that in the field of computer science, published preprints feature adequate revisions, multiple authorship, detailed abstract and introduction, extensive and authoritative references and available source code.