InvAASTCluster: On Applying Invariant-Based Program Clustering to Introductory Programming Assignments

TL;DR

InvAASTCluster introduces a novel program clustering method using invariants and anonymized syntax trees, improving clustering quality and repair success in introductory programming assignments.

Contribution

The paper presents InvAASTCluster, a new invariant-based program clustering approach that outperforms syntax-based methods and enhances program repair effectiveness.

Findings

Outperforms syntax-based clustering in grouping correct IPAs.

Increases repair success rate by approximately 13%.

Reduces time required for program repair.

Abstract

Due to the vast number of students enrolled in programming courses, there has been an increasing number of automated program repair techniques focused on introductory programming assignments (IPAs). Typically, such techniques use program clustering to take advantage of previous correct student implementations to repair a new incorrect submission. These repair techniques use clustering methods since analyzing all available correct submissions to repair a program is not feasible. However, conventional clustering methods rely on program representations based on features such as abstract syntax trees (ASTs), syntax, control flow, and data flow. This paper proposes InvAASTCluster, a novel approach for program clustering that uses dynamically generated program invariants to cluster semantically equivalent IPAs. InvAASTCluster's program representation uses a combination of the program's semantics, through its invariants, and its structure through its anonymized abstract syntax tree (AASTs). Invariants denote conditions that must remain true during program execution, while AASTs are ASTs devoid of variable and function names, retaining only their types. Our experiments show that the proposed program representation outperforms syntax-based representations when clustering a set of correct IPAs. Furthermore, we integrate InvAASTCluster into a state-of-the-art clustering-based program repair tool. Our results show that InvAASTCluster advances the current state-of-the-art when used by clustering-based repair tools by repairing around 13% more students' programs, in a shorter amount of time.