DRAGON: Robust Classification for Very Large Collections of Software Repositories

TL;DR

DRAGON is a robust, scalable classifier for large software repositories that effectively uses lightweight signals like file names and README files, outperforming existing methods especially when documentation is sparse.

Contribution

The paper introduces DRAGON, a novel repository classification approach that operates effectively without relying heavily on README files, suitable for large, diverse collections.

Findings

DRAGON improves F1@5 from 54.8% to 60.8%.

Performance degrades only 6% without README files.

The authors release a dataset of 825,000 repositories for future research.

Abstract

The ability to automatically classify source code repositories with ''topics'' that reflect their content and purpose is very useful, especially when navigating or searching through large software collections. However, existing approaches often rely heavily on README files and other metadata, which are frequently missing, limiting their applicability in real-world large-scale settings. We present DRAGON, a repository classifier designed for very large and diverse software collections. It operates entirely on lightweight signals commonly stored in version control systems: file and directory names, and optionally the README when available. In repository classification at scale, DRAGON improves F1@5 from 54.8% to 60.8%, surpassing the state of the art. DRAGON remains effective even when README files are absent, with performance degrading by only 6% w.r.t. when they are present. This robustness makes it practical for real-world settings where documentation is sparse or inconsistent. Furthermore, many of the remaining classification errors are near misses, where predicted labels are semantically close to the correct topics. This property increases the practical value of the predictions in real-world software collections, where suggesting a few related topics can still guide search and discovery. As a byproduct of developing DRAGON, we also release the largest open dataset to date for repository classification, consisting of 825 thousand repositories with associated ground-truth topics, sourced from the Software Heritage archive, providing a foundation for future large-scale and language-agnostic research on software repository understanding.