Adaptive Sign Error Control
Chaoyu Yu, Peter D. Hoff

TL;DR
This paper introduces two adaptive procedures for controlling the sign error rate in multiple testing, one conservative and one empirical Bayes-based, improving sign inference accuracy without relying on traditional error rate controls.
Contribution
It proposes novel adaptive methods specifically targeting sign error control, including a distribution-free approach and an asymptotically optimal empirical Bayes procedure.
Findings
The conservative method controls sign error rate without distributional assumptions.
The empirical Bayes method achieves asymptotic optimality under correct model specification.
An adaptive procedure minimizes sign errors at fixed type I error rates.
Abstract
In multiple testing scenarios, typically the sign of a parameter is inferred when its estimate exceeds some significance threshold in absolute value. Typically, the significance threshold is chosen to control the experimentwise type I error rate, family-wise type I error rate or the false discovery rate. However, controlling these error rates does not explicitly control the sign error rate. In this paper, we propose two procedures for adaptively selecting an experimentwise significance threshold in order to control the sign error rate. The first controls the sign error rate conservatively, without any distributional assumptions on the parameters of interest. The second is an empirical Bayes procedure, and achieves optimal performance asymptotically when a model for the distribution of the parameters is correctly specified. We also discuss an adaptive procedure to minimize the sign error…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Methods in Clinical Trials · Optimal Experimental Design Methods · Statistical Methods and Bayesian Inference
