MiRAGE: Misconception Detection with Retrieval-Guided Multi-Stage Reasoning and Ensemble Fusion

TL;DR

MiRAGE is a novel framework that combines retrieval, multi-stage reasoning, and ensemble fusion to improve automated misconception detection in mathematics, achieving high accuracy and interpretability.

Contribution

It introduces a three-stage process integrating retrieval, reasoning, and reranking, enhancing misconception detection beyond existing methods.

Findings

Achieves Mean Average Precision scores of 0.82/0.92/0.93 at levels 1/3/5.

Outperforms individual modules on mathematics datasets.

Reduces reliance on large-scale language models.

Abstract

Detecting student misconceptions in open-ended responses is a longstanding challenge, demanding semantic precision and logical reasoning. We propose MiRAGE - Misconception Detection with Retrieval-Guided Multi-Stage Reasoning and Ensemble Fusion, a novel framework for automated misconception detection in mathematics. MiRAGE operates in three stages: (1) a Retrieval module narrows a large candidate pool to a semantically relevant subset; (2) a Reasoning module employs chain-of-thought generation to expose logical inconsistencies in student solutions; and (3) a Reranking module refines predictions by aligning them with the reasoning. These components are unified through an ensemble-fusion strategy that enhances robustness and interpretability. On mathematics datasets, MiRAGE achieves Mean Average Precision scores of 0.82/0.92/0.93 at levels 1/3/5, consistently outperforming individual modules. By coupling retrieval guidance with multi-stage reasoning, MiRAGE reduces dependence on large-scale language models while delivering a scalable and effective solution for educational assessment.