TraceDiff: Debugging Unexpected Code Behavior Using Trace Divergences

TL;DR

TraceDiff is a debugging tool that uses trace divergence visualization to help students identify and fix bugs in programming assignments, improving upon existing visual debugging methods.

Contribution

The paper introduces TraceDiff, a novel debugging system that combines program synthesis and visualization to highlight trace divergences for educational purposes.

Findings

Students using TraceDiff detected bugs more effectively.

TraceDiff outperformed existing visual debugging tools.

The system aligns with pedagogical guidelines for feedback design.

Abstract

Recent advances in program synthesis offer means to automatically debug student submissions and generate personalized feedback in massive programming classrooms. When automatically generating feedback for programming assignments, a key challenge is designing pedagogically useful hints that are as effective as the manual feedback given by teachers. Through an analysis of teachers' hint-giving practices in 132 online Q&A posts, we establish three design guidelines that an effective feedback design should follow. Based on these guidelines, we develop a feedback system that leverages both program synthesis and visualization techniques. Our system compares the dynamic code execution of both incorrect and fixed code and highlights how the error leads to a difference in behavior and where the incorrect code trace diverges from the expected solution. Results from our study suggest that our system enables students to detect and fix bugs that are not caught by students using another existing visual debugging tool.