Automated Identification of Logical Errors in Programs: Advancing Scalable Analysis of Student Misconceptions

TL;DR

This paper introduces a scalable, explainable neural framework for automatically detecting logical errors in student code, aiming to improve understanding of student misconceptions in programming education.

Contribution

The paper presents the Subtree-based Attention Neural Network (SANN), a novel explainable AST embedding model for identifying logical errors in student programs.

Findings

Framework accurately detects logical errors in student code.

Provides deeper insights into students' learning processes.

Enhances programming education through targeted feedback.

Abstract

In Computer Science (CS) education, understanding factors contributing to students' programming difficulties is crucial for effective learning support. By identifying specific issues students face, educators can provide targeted assistance to help them overcome obstacles and improve learning outcomes. While identifying sources of struggle, such as misconceptions, in real-time can be challenging in current educational practices, analyzing logical errors in students' code can offer valuable insights. This paper presents a scalable framework for automatically detecting logical errors in students' programming solutions. Our framework is based on an explainable Abstract Syntax Tree (AST) embedding model, the Subtree-based Attention Neural Network (SANN), that identifies the structural components of programs containing logical errors. We conducted a series of experiments to evaluate its effectiveness, and the results suggest that our framework can accurately capture students' logical errors and, more importantly, provide us with deeper insights into their learning processes, offering a valuable tool for enhancing programming education.