Toward a valid instrument for measuring physics quantitative literacy

TL;DR

The paper introduces the Physics Inventory of Quantitative Literacy (PIQL), a validated tool designed to assess students' quantitative skills in introductory physics, using comprehensive statistical analyses to ensure its reliability.

Contribution

It presents the development and validation process of the PIQL, including item analysis and the impact of multiple-response questions, establishing it as a reliable assessment instrument.

Findings

PIQL is a valid and reliable instrument for measuring physics quantitative literacy.

Classical test theory and factor analysis informed item selection and modification.

The instrument is effective in calculus-based introductory physics courses.

Abstract

We have developed the Physics Inventory of Quantitative Literacy (PIQL) as a tool to measure students' quantitative literacy in the context of introductory physics topics. We present the results from various quantitative analyses used to establish the validity of both the individual items and the PIQL as a whole. We show how examining the results from classical test theory analyses, factor analysis, and item response curves informed decisions regarding the inclusion, removal, or modification of items. We also discuss how the choice to include multiple-choice/multiple-response items has informed both our choices for analyses and the interpretations of their results. We are confident that the most recent version of the PIQL is a valid and reliable instrument for measuring students' physics quantitative literacy in calculus-based introductory physics courses at our primary research site. More data are needed to establish its validity for use at other institutions and in other courses.