Combining model tracing and constraint-based modeling for multistep strategy diagnoses

TL;DR

This paper introduces a novel approach that combines model tracing and constraint-based modeling to improve diagnosis of student strategies in multistep problem-solving tasks, demonstrated on quadratic equations.

Contribution

It proposes a merged framework for diagnosis that handles combined steps and aligns well with human teacher assessments.

Findings

System diagnosis matched teachers in all cases

Approach effectively detects strategy deviations

Supports diagnosis of combined-step inputs

Abstract

Model tracing and constraint-based modeling are two approaches to diagnose student input in stepwise tasks. Model tracing supports identifying consecutive problem-solving steps taken by a student, whereas constraint-based modeling supports student input diagnosis even when several steps are combined into one step. We propose an approach that merges both paradigms. By defining constraints as properties that a student input has in common with a step of a strategy, it is possible to provide a diagnosis when a student deviates from a strategy even when the student combines several steps. In this study we explore the design of a system for multistep strategy diagnoses, and evaluate these diagnoses. As a proof of concept, we generate diagnoses for an existing dataset containing steps students take when solving quadratic equations (n=2136). To compare with human diagnoses, two teachers coded a random sample of deviations (n=70) and applications of the strategy (n=70). Results show that that the system diagnosis aligned with the teacher coding in all of the 140 student steps.