What are we assessing? An analysis of the most common concept inventories in physics

TL;DR

This paper analyzes how well common physics concept inventories align with modern educational goals, finding they mainly assess core ideas but lack emphasis on scientific practices and crosscutting concepts.

Contribution

It introduces an analysis of existing assessments using the 3D-LAP framework, highlighting gaps in evaluating scientific practices and crosscutting concepts.

Findings

Concept inventories mainly assess core ideas.

Limited assessment of scientific practices.

Few crosscutting concepts are evaluated.

Abstract

Assessing student learning is a cornerstone of educational practice. Standardized assessments have played a significant role in the development of instruction, curricula, and educational spaces in college physics. However, the use of these assessments to evaluate student learning is only productive if they continue to align with our learning goals. Recently, there have been calls to elevate the process of science ("scientific practices") to the same level of importance and emphasis as the concepts of physics ("core ideas" and "crosscutting concepts"). We use the recently developed 3D-LAP to investigate how well the most commonly used standardized assessments in introductory physics (i.e. concept inventories) align with this modern understanding of physics education's learning goals. We find that many of the questions on concept inventories do elicit evidence of student understanding of core ideas, but do not have the potential to elicit evidence of scientific practices or crosscutting concepts. Furthermore, we find that the individual scientific practices and crosscutting concepts that are assessed using these tools are limited to a select few. We discuss the implications that these findings have on designing and testing curricula and instruction both in the past and for the future.