Adaptive Sign Error Control

TL;DR

This paper introduces two adaptive procedures for controlling the sign error rate in multiple testing, one conservative and one empirical Bayes-based, improving sign inference accuracy without relying on traditional error rate controls.

Contribution

It proposes novel adaptive methods specifically targeting sign error control, including a distribution-free approach and an asymptotically optimal empirical Bayes procedure.

Findings

The conservative method controls sign error rate without distributional assumptions.

The empirical Bayes method achieves asymptotic optimality under correct model specification.

An adaptive procedure minimizes sign errors at fixed type I error rates.

Abstract

In multiple testing scenarios, typically the sign of a parameter is inferred when its estimate exceeds some significance threshold in absolute value. Typically, the significance threshold is chosen to control the experimentwise type I error rate, family-wise type I error rate or the false discovery rate. However, controlling these error rates does not explicitly control the sign error rate. In this paper, we propose two procedures for adaptively selecting an experimentwise significance threshold in order to control the sign error rate. The first controls the sign error rate conservatively, without any distributional assumptions on the parameters of interest. The second is an empirical Bayes procedure, and achieves optimal performance asymptotically when a model for the distribution of the parameters is correctly specified. We also discuss an adaptive procedure to minimize the sign error rate when the experimentwise type I error rate is held fixed.