A Survey on Feedback Types in Automated Programming Assessment Systems
Eduard Frankford, Tobias Antensteiner, Michael Vierhauser, Clemens Sauerwein, Vivien Wallner, Iris Groher, Reinhold Pl\"osch, Ruth Breu

TL;DR
This paper surveys feedback types in Automated Programming Assessment Systems (APASs), comparing traditional and AI-enhanced feedback methods, and finds that combining unit tests with AI feedback improves student performance.
Contribution
It provides a comprehensive overview of feedback mechanisms in APASs and presents empirical evidence on the effectiveness of AI-driven feedback in programming education.
Findings
Students find unit test feedback most helpful
AI-generated feedback improves student performance
Combining feedback methods optimizes learning outcomes
Abstract
With the recent rapid increase in digitization across all major industries, acquiring programming skills has increased the demand for introductory programming courses. This has further resulted in universities integrating programming courses into a wide range of curricula, including not only technical studies but also business and management fields of study. Consequently, additional resources are needed for teaching, grading, and tutoring students with diverse educational backgrounds and skills. As part of this, Automated Programming Assessment Systems (APASs) have emerged, providing scalable and high-quality assessment systems with efficient evaluation and instant feedback. Commonly, APASs heavily rely on predefined unit tests for generating feedback, often limiting the scope and level of detail of feedback that can be provided to students. With the rise of Large Language Models…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTeaching and Learning Programming · Student Assessment and Feedback · Intelligent Tutoring Systems and Adaptive Learning
