Data-Mining Textual Responses to Uncover Misconception Patterns

TL;DR

This paper introduces a natural language processing framework that automatically detects and classifies student misconceptions from textual responses, enabling scalable and targeted feedback in education.

Contribution

It presents a probabilistic model for analyzing student responses to identify common misconceptions without manual enumeration.

Findings

Framework outperforms baseline in classifying misconceptions

Automatically detects common misconceptions across responses

Scalable approach suitable for large classes

Abstract

An important, yet largely unstudied, problem in student data analysis is to detect misconceptions from students' responses to open-response questions. Misconception detection enables instructors to deliver more targeted feedback on the misconceptions exhibited by many students in their class, thus improving the quality of instruction. In this paper, we propose a new natural language processing-based framework to detect the common misconceptions among students' textual responses to short-answer questions. We propose a probabilistic model for students' textual responses involving misconceptions and experimentally validate it on a real-world student-response dataset. Experimental results show that our proposed framework excels at classifying whether a response exhibits one or more misconceptions. More importantly, it can also automatically detect the common misconceptions exhibited across responses from multiple students to multiple questions; this property is especially important at large scale, since instructors will no longer need to manually specify all possible misconceptions that students might exhibit.