Testing in the Evolving World of DL Systems:Insights from Python GitHub Projects

TL;DR

This paper analyzes testing practices in open-source Deep Learning projects on GitHub, highlighting the adoption, types, and evolution of testing methodologies to improve project quality and reliability.

Contribution

It provides the first comprehensive analysis of testing practices and their evolution in GitHub-hosted DL projects, offering valuable insights for practitioners.

Findings

Testing adoption is increasing in DL projects.

Unit and integration tests are most common.

Test suite growth correlates with project maturity.

Abstract

In the ever-evolving field of Deep Learning (DL), ensuring project quality and reliability remains a crucial challenge. This research investigates testing practices within DL projects in GitHub. It quantifies the adoption of testing methodologies, focusing on aspects like test automation, the types of tests (e.g., unit, integration, and system), test suite growth rate, and evolution of testing practices across different project versions. We analyze a subset of 300 carefully selected repositories based on quantitative and qualitative criteria. This study reports insights on the prevalence of testing practices in DL projects within the open-source community.