Comprehension Debt in GenAI-Assisted Software Engineering Projects

TL;DR

This study explores how GenAI tools in software engineering can lead to Comprehension Debt, a socio-cognitive risk affecting team understanding, with patterns identified from student diaries and strategies for mitigation.

Contribution

It introduces the concept of Comprehension Debt in GenAI-assisted projects, identifies four accumulation patterns, and suggests pedagogical strategies for mitigation.

Findings

Four CD accumulation patterns identified: black-box acceptance, context mismatch, dependency atrophy, verification bypass.

GenAI can serve as a comprehension scaffold to reduce CD.

CD is a socio-cognitive form of debt, distinct from traditional technical debt.

Abstract

Generative Artificial Intelligence (GenAI) tools (e.g., ChatGPT, Calude) have rapidly become integral to software development. These tools are especially attractive to students, as they can reduce cognitive load. However, their adoption also introduces a socio-cognitive risk: the accumulation of Comprehension Debt (CD). CD refers to the growing gap between what a development team knows about its codebase and what it actually needs to understand in order to maintain and modify it effectively. This qualitative study investigate how GenAI tools contribute to CD in the context of an undergraduate software engineering project. Our study is based on 621 reflective diaries from 207 students over eight weeks. We identify four CD accumulation patterns and one mitigating pattern in students' use of GenAI tools. The four accumulation patterns include: (1) AI-as-black-box code acceptance, (2) context-mismatch debt, (3) dependency-induced atrophy, and (4) verification-bypass. In contrast, the mitigating pattern involves students using GenAI as a comprehension scaffold, allowing them to build a deeper understanding of the code. We argue that CD is distinct from traditional technical debt because it resides in the collective cognition of development teams rather than in the codebase itself. Our findings highlight the need for explicit pedagogical strategies to mitigate CD in software engineering education, emphasizing verification practices, structured retrospectives, and active learning assessments.