FIXME: Synchronize with Database An Empirical Study of Data Access Self-Admitted Technical Debt

TL;DR

This empirical study investigates self-admitted technical debt in data-intensive systems, analyzing its prevalence, evolution, and composition across numerous open-source projects to inform better debt management strategies.

Contribution

It provides a large-scale analysis of data-access SATD, identifying new categories and revealing insights into their introduction and removal dynamics.

Findings

Most SATDs are introduced later in development cycles.

Bug fixing and refactoring are primary reasons for SATD introduction.

Identified 15 new SATD categories, 11 specific to database access.

Abstract

Developers sometimes choose design and implementation shortcuts due to the pressure from tight release schedules. However, shortcuts introduce technical debt that increases as the software evolves. The debt needs to be repaid as fast as possible to minimize its impact on software development and software quality. Sometimes, technical debt is admitted by developers in comments and commit messages. Such debt is known as self-admitted technical debt (SATD). In data-intensive systems, where data manipulation is a critical functionality, the presence of SATD in the data access logic could seriously harm performance and maintainability. Understanding the composition and distribution of the SATDs across software systems and their evolution could provide insights into managing technical debt efficiently. We present a large-scale empirical study on the prevalence, composition, and evolution of SATD in data-intensive systems. We analyzed 83 open-source systems relying on relational databases as well as 19 systems relying on NoSQL databases. We detected SATD in source code comments obtained from different snapshots of the subject systems. To understand the evolution dynamics of SATDs, we conducted a survival analysis. Next, we performed a manual analysis of 361 sample data-access SATDs, investigating the composition of data-access SATDs and the reasons behind their introduction and removal. We identified 15 new SATD categories, out of which 11 are specific to database access operations. We found that most of the data-access SATDs are introduced in the later stages of change history rather than at the beginning. We also observed that bug fixing and refactoring are the main reasons behind the introduction of data-access SATDs.