Causal Inference in Educational Systems: A Graphical Modeling Approach

TL;DR

This paper introduces a graphical modeling framework for causal inference in educational systems, accounting for complex feedback, confounders, and sequential interventions to improve evaluation accuracy.

Contribution

It develops new causal inference methods using DAGs for educational systems, incorporating time-varying treatments and confounders, and compares inference techniques like IPTW and g-formula.

Findings

Controlling confounders yields unbiased causal estimates.

IPTW and g-formula have different advantages and limitations.

Graphical models effectively represent complex educational interventions.

Abstract

Educational systems have traditionally been evaluated using cross-sectional studies, namely, examining a pretest, posttest, and single intervention. Although this is a popular approach, it does not model valuable information such as confounding variables, feedback to students, and other real-world deviations of studies from ideal conditions. Moreover, learning inherently is a sequential process and should involve a sequence of interventions. In this paper, we propose various experimental and quasi-experimental designs for educational systems and quantify them using the graphical model and directed acyclic graph (DAG) language. We discuss the applications and limitations of each method in education. Furthermore, we propose to model the education system as time-varying treatments, confounders, and time-varying treatments-confounders feedback. We show that if we control for a sufficient set of confounders and use appropriate inference techniques such as the inverse probability of treatment weighting (IPTW) or g-formula, we can close the backdoor paths and derive the unbiased causal estimate of joint interventions on the outcome. Finally, we compare the g-formula and IPTW performance and discuss the pros and cons of using each method.