Testing Sign Congruence Between Two Parameters

TL;DR

This paper evaluates methods for testing if two parameters share the same sign, proposing a simple, more powerful test, and also discusses an unbiased alternative with less practical appeal, with applications in treatment effects and meta-studies.

Contribution

It introduces a simple, more powerful test for sign congruence and analyzes its properties compared to existing methods, including an unbiased but less practical alternative.

Findings

The recommended test rejects more often than recent proposals.

The unbiased test has exact size control but counterintuitive properties.

Application to existing studies improves p-value accuracy.

Abstract

We test the null hypothesis that two parameters $(\mu_1,\mu_2)$ have the same sign, assuming that (asymptotically) normal estimators $(\hat{\mu}_1,\hat{\mu}_2)$ are available. Examples of this problem include the analysis of heterogeneous treatment effects, causal interpretation of reduced-form estimands, meta-studies, and mediation analysis. A number of tests were recently proposed. We recommend a test that is simple and rejects more often than many of these recent proposals. Like all other tests in the literature, it is conservative if the truth is near $(0,0)$ and therefore also biased. To clarify whether these features are avoidable, we also provide a test that is unbiased and has exact size control on the boundary of the null hypothesis, but which has counterintuitive properties and hence we do not recommend. We use the test to improve p-values in Kowalski (2022) from information contained in that paper's main text and to establish statistical significance of some key estimates in Dippel et al. (2021).