The Rise of GitHub in Scholarly Publications

TL;DR

This paper highlights the increasing use of GitHub and similar platforms in scholarly publications, emphasizing the urgent need for dedicated archiving efforts to preserve research code and context for reproducibility.

Contribution

It provides a quantitative analysis of GHP URI usage in arXiv and PMC, demonstrating the growing reliance on these platforms in scholarly work and the need for specialized archiving solutions.

Findings

GitHub URIs appeared in 1/5 of arXiv publications in 2021.

GHP references increased from 253,590 in 2007 to 76,746 in 2021.

The complexity of GHPs challenges traditional web archiving methods.

Abstract

The definition of scholarly content has expanded to include the data and source code that contribute to a publication. While major archiving efforts to preserve conventional scholarly content, typically in PDFs (e.g., LOCKSS, CLOCKSS, Portico), are underway, no analogous effort has yet emerged to preserve the data and code referenced in those PDFs, particularly the scholarly code hosted online on Git Hosting Platforms (GHPs). Similarly, the Software Heritage Foundation is working to archive public source code, but there is value in archiving the issue threads, pull requests, and wikis that provide important context to the code while maintaining their original URLs. In current implementations, source code and its ephemera are not preserved, which presents a problem for scholarly projects where reproducibility matters. To understand and quantify the scope of this issue, we analyzed the use of GHP URIs in the arXiv and PMC corpora from January 2007 to December 2021. In total, there were 253,590 URIs to GitHub, SourceForge, Bitbucket, and GitLab repositories across the 2.66 million publications in the corpora. We found that GitHub, GitLab, SourceForge, and Bitbucket were collectively linked to 160 times in 2007 and 76,746 times in 2021. In 2021, one out of five publications in the arXiv corpus included a URI to GitHub. The complexity of GHPs like GitHub is not amenable to conventional Web archiving techniques. Therefore, the growing use of GHPs in scholarly publications points to an urgent and growing need for dedicated efforts to archive their holdings in order to preserve research code and its scholarly ephemera.