A Systematic Literature Review of the Use of GenAI Assistants for Code Comprehension: Implications for Computing Education Research and Practice

TL;DR

This systematic review examines how GenAI assistants are used to improve code comprehension in computing education, highlighting their potential, current challenges, and directions for future research.

Contribution

It provides a comprehensive classification of GenAI approaches for code comprehension and synthesizes empirical evidence from 31 recent studies.

Findings

GenAI tools often produce inaccurate explanations

Novice programmers struggle with effective prompt crafting

Empirical evaluations show mixed effectiveness of GenAI in education

Abstract

The ability to comprehend code has long been recognized as an essential skill in software engineering. As programmers lean more heavily on generative artificial intelligence (GenAI) assistants to develop code solutions, it is becoming increasingly important for programmers to comprehend GenAI solutions so that they can verify their appropriateness and properly integrate them into existing code. At the same time, GenAI tools are increasingly being enlisted to provide programmers with tailored explanations of code written both by GenAI and humans. Thus, in computing education, GenAI presents new challenges and opportunities for learners who are trying to comprehend computer programs. To provide computing educators with evidence-based guidance on the use of GenAI to facilitate code comprehension and to identify directions for future research, we present a systematic literature review (SLR) of state-of-the-art approaches and tools that leverage GenAI to enhance code comprehension. Our SLR focuses on 31 studies published between 2022 and 2024. Despite their potential, GenAI assistants often yield inaccurate or unclear explanations, and novice programmers frequently struggle to craft effective prompts, thereby impeding their ability to leverage GenAI to aid code comprehension. Our review classifies GenAI-based approaches and tools, identifies methods used to study them, and summarizes the empirical evaluations of their effectiveness. We consider the implications of our findings for computing education research and practice, and identify directions for future research.