Visualizing changes in student responses using consistency plots

TL;DR

This paper introduces consistency plots, a visualization method that depicts individual student response changes, including solution methods and correctness, over a semester, providing more detailed insights than traditional average-based reports.

Contribution

The paper extends existing escalator diagrams to include solution methods and correctness before and after instruction, offering a richer visualization of student response dynamics.

Findings

Consistency plots reveal detailed response changes at the individual level.

Method captures shifts in solution approaches and correctness.

Applied to mechanics class data from midterm and final exams.

Abstract

Traditional methods of reporting changes in student responses have focused on class-wide averages. Such models hide information about the switches in responses by individual students over the course of a semester. We extend unpublished work by Steven Kanim on "escalator diagrams" which show changes in student responses from correct to incorrect (and vice versa) while representing pre- and post-instruction results on questions. Our extension consists of "consistency plots" in which we represent three forms of data: method of solution and correctness of solution both before and after instruction. Our data are from an intermediate mechanics class, and come from (nearly) identical midterm and final examination questions.