Exploring the Role of Automated Feedback in Programming Education: A Systematic Literature Review

TL;DR

This systematic review analyzes 61 studies on automated feedback in programming education, highlighting current practices, technological advancements, and gaps in supporting higher-order learning and long-term outcomes.

Contribution

It synthesizes empirical research across multiple dimensions, providing a comprehensive overview and identifying areas for future pedagogical and technological development.

Findings

Most systems are fully automated and error-focused.

Recent advances include adaptive features and large language models.

Evaluation mainly emphasizes short-term performance.

Abstract

Automated feedback systems have become increasingly integral to programming education, where learners engage in iterative cycles of code construction, testing, and refinement. Despite its wider integration in practices and technical advancements into AI, research in this area remains fragmented, lacking synthesis across technological and instructional dimensions. This systematic literature review synthesizes 61 empirical studies published by September 2024, offering a conceptually grounded analysis of automated feedback systems across five dimensions: system architecture, pedagogical function, interaction mechanism, contextual deployment, and evaluation approach. Findings reveal that most systems are fully automated, embedded within online platforms, and primarily focused on error detection and code correctness. While recent developments incorporate adaptive features and large language models to enable more personalized and interactive feedback, few systems offer support for higher-order learning processes, interactive components, or learner agency. Moreover, evaluation practices tend to emphasize short-term performance gains, with limited attention to long-term outcomes or instructional integration. These findings call for a reimagining of automated feedback not as a technical add-on for error correction, but as a pedagogical scaffold that supports deeper, adaptive, and interactive learning.