The Nature of Technical Debt in Research Software

TL;DR

This study investigates the nature and impact of technical debt in research software through analysis of code comments and interviews with developers, revealing nine unique types and four influencing themes.

Contribution

It is the first comprehensive multi-method study characterizing technical debt in research software, identifying its types, manifestations, and effects on scientific research.

Findings

Nine types of technical debt unique to research software identified

Four themes influence the manifestation of technical debt

Analysis of 28,000 code comments and developer interviews conducted

Abstract

Research software (also called scientific software) is essential for advancing scientific endeavours. Research software encapsulates complex algorithms and domain-specific knowledge and is a fundamental component of all science. A pervasive challenge in developing research software is technical debt, which can adversely affect reliability, maintainability, and scientific validity. Research software often relies on the initiative of the scientific community for maintenance, requiring diverse expertise in both scientific and software engineering domains. The extent and nature of technical debt in research software are little studied, in particular, what forms it takes, and what the science teams developing this software think about their technical debt. In this paper we describe our multi-method study examining technical debt in research software. We begin by examining instances of self-reported technical debt in research code, examining 28k code comments across nine research software projects. Then, building on our findings, we interview research software engineers and scientists about how this technical debt manifests itself in their experience, and what costs it has for research software and research outputs more generally. We identify nine types of self-admitted technical debt unique to research software, and four themes impacting this technical debt.