Exploring Sustainability in Scientific Software through Code Quality & Test Coverage Metrics

TL;DR

This paper assesses the long-term sustainability of scientific open-source software by analyzing code quality and test coverage metrics, revealing patterns that distinguish sustainable projects from less sustainable ones.

Contribution

It introduces a data-driven approach to evaluate scientific software sustainability through code and test quality metrics, aiding future quality assurance efforts.

Findings

Sustainable projects have higher, more consistent test coverage.

Code-test correlations are clearer in sustainable projects.

Overall, scientific software exhibits low test coverage and high complexity reduces testability.

Abstract

Context: Scientific open-source software (SciOSS) plays a foundational role in research and engineering, yet its long-term sustainability has often been overlooked and remains a significant concern. Objective: This study investigates the long-term sustainability of SciOSS through code and test quality metrics. Method: We analyze CASS Software Portfolio projects, classifying them by sustainability and comparing their code structure, test coverage, and links between code quality and testing across the dataset. Results: Sustainable projects show higher, more consistent test coverage and clearer code-test correlations, while unsustainable ones show weaker patterns. Overall, test coverage is low in scientific software, and high complexity and coupling reduce testability. Conclusion: In this study, we present a practical, data-driven approach for assessing sustainability in scientific software, offering a foundation for evaluating long-term software health and supporting future efforts in quality assurance and sustainability monitoring.