Logic Error Localization in Student Programming Assignments Using Pseudocode and Graph Neural Networks

TL;DR

This paper presents a novel system using pseudocode and graph neural networks to accurately localize logic errors in student programming assignments, improving debugging and learning outcomes.

Contribution

It introduces a graph-based approach leveraging pseudocode to detect and suggest corrections for logic errors at the line level in student code.

Findings

Localization accuracy of 99.2% within top-10 lines

Effective dataset creation for logic errors during syntax correction

Enhanced student debugging and learning process

Abstract

Pseudocode is extensively used in introductory programming courses to instruct computer science students in algorithm design, utilizing natural language to define algorithmic behaviors. This learning approach enables students to convert pseudocode into source code and execute it to verify their algorithms' correctness. This process typically introduces two types of errors: syntax errors and logic errors. Syntax errors are often accompanied by compiler feedback, which helps students identify incorrect lines. In contrast, logic errors are more challenging because they do not trigger compiler errors and lack immediate diagnostic feedback, making them harder to detect and correct. To address this challenge, we developed a system designed to localize logic errors within student programming assignments at the line level. Our approach utilizes pseudocode as a scaffold to build a code-pseudocode graph, connecting symbols from the source code to their pseudocode counterparts. We then employ a graph neural network to both localize and suggest corrections for logic errors. Additionally, we have devised a method to efficiently gather logic-error-prone programs during the syntax error correction process and compile these into a dataset that includes single and multiple line logic errors, complete with indices of the erroneous lines. Our experimental results are promising, demonstrating a localization accuracy of 99.2% for logic errors within the top-10 suspected lines, highlighting the effectiveness of our approach in enhancing students' coding proficiency and error correction skills.