Methods of Causal Inference in Physics Education Research

TL;DR

This paper critically examines recent physics education research papers on causal inference, highlighting significant methodological errors in causal reasoning, variable control, and data imputation.

Contribution

It provides a detailed analysis of errors in current causal inference methods used in physics education research, emphasizing the need for improved methodological rigor.

Findings

Identification of major errors in causal mediation

Incorrect variable control strategies

Faulty data imputation practices

Abstract

Finding good educational policies requires sound estimates of their potential effects. Methods for making such estimates, i.e. finding causal estimands, have made great progress in the last few decades. Nevertheless, serious errors in causal reasoning have been found previously in papers n a leading physics education journal, Physical Review Physics Education Research. Here we examine three more recent papers from that journal that present explicit methods of causal inference. The methods given include major errors, including in identifying causal mediation, choosing variables to control for, and imputing missing data.