Replicating analyses of item response curves using data from the Force and Motion Conceptual Evaluation

TL;DR

This study replicates and extends previous analyses of student response data to the FMCE using item response curves, revealing mostly consistent results and highlighting the pedagogical value of IRCs for understanding student thinking.

Contribution

It provides an independent validation of prior IRC analyses on FMCE data and demonstrates the effectiveness of vector-based comparison methods.

Findings

Most IRC analyses are consistent with previous reports.

Minor differences observed between American and Japanese students.

Highlights pedagogical benefits of using IRCs for analyzing student responses.

Abstract

Ishimoto, Davenport, and Wittmann have previously reported analyses of data from student responses to the Force and Motion Conceptual Evaluation (FMCE), in which they used item response curves (IRCs) to make claims about American and Japanese students' relative likelihood to choose certain incorrect responses to some questions. We have used an independent data set of over 6,500 American students' responses to the FMCE to generate IRCs to test their claims. Converting the IRCs to vectors, we used dot product analysis to compare each response item quantitatively. For most questions, our analyses are consistent with Ishimoto, Davenport, and Wittmann, with some results suggesting more minor differences between American and Japanese students than previously reported. We also highlight the pedagogical advantages of using IRCs to determine the differences in response patterns for different populations to better understand student thinking prior to instruction.