Automating Program Structure Classification

TL;DR

This paper presents machine learning approaches to automatically classify student programming solutions into high-level structures, aiding educators in analyzing student learning processes efficiently.

Contribution

It introduces supervised machine learning models, including a nearest-neighbors and RNN, for classifying student programs into predefined structural categories.

Findings

Achieved 91% accuracy on classifying 108 student programs

Compared effectiveness of syntax tree edit distance and RNN models

Explored model generality, trade-offs, and failure cases

Abstract

When students write programs, their program structure provides insight into their learning process. However, analyzing program structure by hand is time-consuming, and teachers need better tools for computer-assisted exploration of student solutions. As a first step towards an education-oriented program analysis toolkit, we show how supervised machine learning methods can automatically classify student programs into a predetermined set of high-level structures. We evaluate two models on classifying student solutions to the Rainfall problem: a nearest-neighbors classifier using syntax tree edit distance and a recurrent neural network. We demonstrate that these models can achieve 91% classification accuracy when trained on 108 programs. We further explore the generality, trade-offs, and failure cases of each model.