On the Use of ArXiv as a Dataset

TL;DR

This paper introduces a standardized pipeline to access and analyze arXiv's extensive dataset, including a large citation graph and full-text corpus, to facilitate benchmarking and development of advanced models.

Contribution

It provides the first comprehensive pipeline for standardized access to arXiv data, enabling large-scale analysis and benchmarking of models on multi-modal scientific literature.

Findings

Constructed a 6.7 million edge citation graph.

Compiled an 11 billion word full-text corpus.

Presented baseline classification results.

Abstract

The arXiv has collected 1.5 million pre-print articles over 28 years, hosting literature from scientific fields including Physics, Mathematics, and Computer Science. Each pre-print features text, figures, authors, citations, categories, and other metadata. These rich, multi-modal features, combined with the natural graph structure---created by citation, affiliation, and co-authorship---makes the arXiv an exciting candidate for benchmarking next-generation models. Here we take the first necessary steps toward this goal, by providing a pipeline which standardizes and simplifies access to the arXiv's publicly available data. We use this pipeline to extract and analyze a 6.7 million edge citation graph, with an 11 billion word corpus of full-text research articles. We present some baseline classification results, and motivate application of more exciting generative graph models.