A Framework for Evaluating Statistical Models in Physics Education Research

TL;DR

This paper presents a comprehensive framework for evaluating statistical models in physics education research, emphasizing data management, model assessment, and effective communication, demonstrated through survey data analysis.

Contribution

It introduces a novel framework tailored for PER that integrates machine learning, data handling, and evaluation, enhancing research rigor and reproducibility.

Findings

Framework effectively evaluates models in PER context

Demonstrates utility with survey data analysis

Supports improved interpretation of educational data

Abstract

Across the field of education research there has been an increased focus on the development, critique, and evaluation of statistical methods and data usage due to recently created, very large data sets and machine learning techniques. In physics education research (PER), this increased focus has recently been shown through the 2019 Physical Review PER Focused Collection examining quantitative methods in PER. Quantitative PER has provided strong arguments for reforming courses by including interactive engagement, demonstrated that students often move away from scientist-like views due to science education, and has injected robust assessment into the physics classroom via concept inventories. The work presented here examines the impact that machine learning may have on physics education research, presents a framework for the entire process including data management, model evaluation, and results communication, and demonstrates the utility of this framework through the analysis of two types of survey data.