On Policy Recommendations from Causal Inference in Physics Education Research

TL;DR

This paper discusses the importance of clear causal inference in physics education research to ensure valid policy recommendations, highlighting common issues with causal assumptions and the use of causal graphs.

Contribution

It critically examines how causal inferences are made in physics education research and emphasizes the need for explicit causal hypotheses to improve policy recommendations.

Findings

Causal conclusions often depend on unstated assumptions.

Misinterpretation of causal graphs can lead to invalid policy advice.

Explicit causal hypotheses are essential for valid inferences.

Abstract

Sound educational policy recommendations require valid estimates of causal effects, but observational studies in physics education research sometimes have loosely specified causal hypotheses. The connections between the observational data and the explicit or implicit causal conclusions are sometimes misstated. The link between the causal conclusions reached and the policy recommendations made is also sometimes loose. Causal graphs are used to illustrate these issues in several papers from Physical Review Physics Education Research. For example, the core causal conclusion of one paper rests entirely on the choice of a causal direction although an unstated plausible alternative gives an exactly equal fit to the data.