A Response to Recent Critiques of Hainmueller, Mummolo and Xu (2019) on Estimating Conditional Relationships

TL;DR

This paper defends the methodology of Hainmueller, Mummolo and Xu (2019) against recent critiques, clarifying misconceptions about modeling nonlinear relationships and emphasizing correct estimation of interaction effects.

Contribution

The paper refutes recent critiques by clarifying the proper causal estimand, demonstrating the effectiveness of the kernel estimator, and providing updated guidance for estimating interaction effects.

Findings

Kernel estimator recovers true causal effects in critiques' scenarios.

Misinterpretation of effects due to incorrect benchmarks.

GAMs are useful for exploration but not for estimating marginal effects.

Abstract

Simonsohn (2024a) and Simonsohn (2024b) critique Hainmueller, Mummolo and Xu (2019, HMX), arguing that failing to model nonlinear relationships between the treatment and moderator leads to biased marginal effect estimates and uncontrolled Type-I error rates. While these critiques highlight the issue of under-modeling nonlinearity in applied research, they are fundamentally flawed in several key ways. First, the causal estimand for interaction effects and the necessary identifying assumptions are not clearly defined in these critiques. Once properly stated, the critiques no longer hold. Second, the kernel estimator HMX proposes recovers the true causal effects in the scenarios presented in these recent critiques, which compared effects to the wrong benchmark, producing misleading conclusions. Third, while Generalized Additive Models (GAM) can be a useful exploratory tool (as acknowledged in HMX), they are not designed to estimate marginal effects, and better alternatives exist, particularly in the presence of additional covariates. Our response aims to clarify these misconceptions and provide updated recommendations for researchers studying interaction effects through the estimation of conditional marginal effects.