Regression-based causal inference with factorial experiments: estimands, model specifications, and design-based properties

TL;DR

This paper analyzes the sampling properties of factor-based regression estimators in factorial experiments, establishing their theoretical validity, efficiency trade-offs, and proposing a unified framework for factorial effect estimation.

Contribution

It provides a comprehensive theoretical foundation for regression-based causal inference in factorial designs, including new results on estimator properties and a unified effect definition.

Findings

Robust standard errors are appropriate for Wald inference.

Saturated models have less bias but higher variance than unsaturated models.

Using parsimonious models improves efficiency when nuisance effects are absent.

Abstract

Factorial designs are widely used due to their ability to accommodate multiple factors simultaneously. The factor-based regression with main effects and some interactions is the dominant strategy for downstream data analysis, delivering point estimators and standard errors via one single regression. Justification of these convenient estimators from the design-based perspective requires quantifying their sampling properties under the assignment mechanism conditioning on the potential outcomes. To this end, we derive the sampling properties of the factor-based regression estimators from both saturated and unsaturated models, and demonstrate the appropriateness of the robust standard errors for the Wald-type inference. We then quantify the bias-variance trade-off between the saturated and unsaturated models from the design-based perspective, and establish a novel design-based Gauss--Markov theorem that ensures the latter's gain in efficiency when the nuisance effects omitted indeed do not exist. As a byproduct of the process, we unify the definitions of factorial effects in various literatures and propose a location-shift strategy for their direct estimation from factor-based regressions. Our theory and simulation suggest using factor-based inference for general factorial effects, preferably with parsimonious specifications in accordance with the prior knowledge of zero nuisance effects.