Model-free Sign Estimation for High-Throughput Screenings

TL;DR

This paper introduces a minimal-assumption, model-free method to estimate the signs of treatment effects in high-throughput screenings, effectively controlling error rates and outperforming existing practices in simulations and real data.

Contribution

It proposes a novel, model-free approach for sign estimation that better controls errors compared to traditional p-value based methods in high-throughput screening data.

Findings

The method controls sign misestimation effectively.

It outperforms Benjamini-Hochberg in simulations.

It provides more accurate sign inference on real data.

Abstract

In high-throughput screenings, it is common to estimate the effects of many treatments using a small number of independent trials of each. Because little is known about the distributional properties of the measurements from these trials, it is challenging to identify plausible assumptions that can serve as a basis for inferential statistics in this setting. In this article, we develop a method based on minimal assumptions to infer signs of treatment effects (positive or negative). The proposed method controls the number of misestimated signs by using the number of sign disagreements between measurements of the same treatment as a proxy for the number of sign errors. In simulations, the proposed method compares favorably with the Benjamini-Hochberg procedure applied to invalid $p$-values, which is currently considered best practice for many high-throughput screenings. For real data from the L1000 cell-perturbation platform, the proposed method outperforms existing practices, which fail to control error at the nominal level in some cases and are needlessly conservative in others.