Math Operation Embeddings for Open-ended Solution Analysis and Feedback

TL;DR

This paper introduces a scalable method for analyzing student solutions to math problems by embedding math operations into a vector space, enabling automatic feedback without manual error modeling.

Contribution

It proposes a novel math operation embedding technique that generalizes error detection and feedback selection across diverse math questions.

Findings

Embeddings effectively identify student-intended math operations.

The approach generalizes well across different datasets.

It reduces manual effort in creating cognitive models.

Abstract

Feedback on student answers and even during intermediate steps in their solutions to open-ended questions is an important element in math education. Such feedback can help students correct their errors and ultimately lead to improved learning outcomes. Most existing approaches for automated student solution analysis and feedback require manually constructing cognitive models and anticipating student errors for each question. This process requires significant human effort and does not scale to most questions used in homework and practices that do not come with this information. In this paper, we analyze students' step-by-step solution processes to equation solving questions in an attempt to scale up error diagnostics and feedback mechanisms developed for a small number of questions to a much larger number of questions. Leveraging a recent math expression encoding method, we represent each math operation applied in solution steps as a transition in the math embedding vector space. We use a dataset that contains student solution steps in the Cognitive Tutor system to learn implicit and explicit representations of math operations. We explore whether these representations can i) identify math operations a student intends to perform in each solution step, regardless of whether they did it correctly or not, and ii) select the appropriate feedback type for incorrect steps. Experimental results show that our learned math operation representations generalize well across different data distributions.