Analyzing Force Concept Inventory with Item Response Theory

TL;DR

This paper applies Item Response Theory to analyze the Force Concept Inventory, providing a detailed measurement metric that enhances assessment accuracy and comparability in physics education.

Contribution

It introduces an IRT-based analysis of the FCI, offering a new detailed measurement metric for physics assessments based on data from over 2800 students.

Findings

IRT provides a nuanced understanding of item characteristics.

The analysis yields a comprehensive measurement metric for FCI.

Results facilitate improved assessment and interpretation in physics education.

Abstract

Item Response Theory (IRT) is a popular assessment method used in education measurement, which builds on an assumption of a probability framework connecting students' innate ability and their actual performances on test items. The model transforms students' raw test scores through a nonlinear regression process into a scaled proficiency rating, which can be used to compare results obtained with different test questions. IRT also provides a theoretical approach to address ceiling effect and guessing. We applied IRT to analyze the Force Concept Inventory (FCI). The data was collected from 2802 students taking intro level mechanics courses at The Ohio State University. The data was analyzed with a 3-parameter item response model for multiple choice questions. We describe the procedures of the analysis and discuss the results and the interpretations. The analysis outcomes are compiled to provide a detailed IRT measurement metric of the FCI, which can be easily referenced and used by teachers and researchers for a range of assessment applications.