Applying causal inference to inform early-childhood policy from administrative data

TL;DR

This paper advocates for the use of causal inference methods on administrative data to improve policy decisions, demonstrating their advantages over traditional descriptive analyses through simulations and case studies in education.

Contribution

It provides practical guidance on applying causal diagrams and methods in policy analysis, emphasizing their importance for accurate impact estimation and bias reduction.

Findings

Causal diagrams help in selecting relevant variables.

Including extraneous variables can introduce bias.

Causal methods improve policy impact estimates.

Abstract

Improving public policy is one of the key roles of governments, and they can do this in an evidence-based way using administrative data. Causal inference for observational data improves on current practice of using descriptive or predictive analyses to inform policy decisions. Causal inference allows analysts to estimate the impact a policy change would have on the population if the encoded assumptions about the data generation process are valid. In this paper, we discuss the importance of causal analysis methods when analysing data to inform policy decisions. We take the education sector as a case study and provide examples of when to use a causal analysis. We use simulation to demonstrate the vital role causal diagrams play in variable selection and how bias can be introduced if extraneous variables are included in the model. Our exploration provides clear evidence for the utility of causal methods and practical examples of how to conduct such analyses. The paper promotes the incorporation of these methods in policy both for improved educational outcomes and scientific understanding.