When effects cannot be estimated: redefining estimands to understand the effects of naloxone access laws

TL;DR

This paper addresses the challenge of positivity violations in health policy evaluation, proposing a modified treatment policy approach to estimate effects of Naloxone Access Laws despite these issues.

Contribution

It introduces a framework for redefining estimands using modified treatment policies to overcome positivity violations in longitudinal health policy studies.

Findings

Empirical evidence of positivity violations in Naloxone law effect estimation.

Modified treatment policies enable effect estimation despite positivity issues.

Application to Naloxone laws shows practical utility of the approach.

Abstract

Violations of the positivity assumption (also called the common support condition) challenge health policy research, and can result in significant bias, large variance, and invalid inference. We define positivity in the single- and multiple-timepoint (i.e., longitudinal) health policy evaluation setting, and discuss real-world threats to positivity. We show empirical evidence of the practical positivity violations that can result when attempting to estimate effects of health policies (in this case, Naloxone Access Laws). In such scenarios, an alternative is to estimate the effect of a shift in law enactment (e.g., the effect if enactment had been delayed by some number of years). Such an effect corresponds to what is called a modified treatment policy, and dramatically weakens the required positivity assumption, thereby offering a means to estimate policy effects even in scenarios with serious positivity problems. We apply the approach to define and estimate longitudinal effects of Naloxone Access Laws on opioid overdose rates.