Autograder+: A Multi-Faceted AI Framework for Rich Pedagogical Feedback in Programming Education

TL;DR

Autograder+ enhances programming education by providing scalable, meaningful feedback and visualization tools using AI, transforming autograders into formative learning aids that support student understanding and instructor guidance.

Contribution

It introduces a novel AI framework that generates pedagogically aligned feedback and visualizes student code to improve programming education.

Findings

Feedback aligns strongly with instructor comments

Code embeddings enable meaningful clustering of solutions

System reduces instructor workload while enhancing learning

Abstract

The rapid growth of programming education has outpaced traditional assessment tools, leaving faculty with limited means to provide meaningful, scalable feedback. Conventional autograders, while efficient, act as black-box systems that simply return pass/fail results, offering little insight into student thinking or learning needs. Autograder+ is designed to shift autograding from a purely summative process to a formative learning experience. It introduces two key capabilities: automated feedback generation using a fine-tuned Large Language Model, and visualization of student code submissions to uncover learning patterns. The model is fine-tuned on curated student code and expert feedback to ensure pedagogically aligned, context-aware guidance. In evaluation across 600 student submissions from multiple programming tasks, the system produced feedback with strong semantic alignment to instructor comments. For visualization, contrastively learned code embeddings trained on 1,000 annotated submissions enable grouping solutions into meaningful clusters based on functionality and approach. The system also supports prompt-pooling, allowing instructors to guide feedback style through selected prompt templates. By integrating AI-driven feedback, semantic clustering, and interactive visualization, Autograder+ reduces instructor workload while supporting targeted instruction and promoting stronger learning outcomes.