Decentralized Nonparametric Multiple Testing
Subhadeep Mukhopadhyay

TL;DR
This paper introduces a decentralized approach for large-scale multiple testing that allows independent processing on distributed nodes without data exchange, leveraging a nonparametric statistical framework to simplify the algorithm design.
Contribution
It proposes a novel decentralized nonparametric multiple testing method suitable for distributed big data environments, filling a gap in existing literature.
Findings
Enables independent parallel testing without data exchange.
Simplifies large-scale inference using a new computing model.
Addresses a previously unexplored problem in distributed multiple testing.
Abstract
Consider a big data multiple testing task, where, due to storage and computational bottlenecks, one is given a very large collection of p-values by splitting into manageable chunks and distributing over thousands of computer nodes. This paper is concerned with the following question: How can we find the full data multiple testing solution by operating completely independently on individual machines in parallel, without any data exchange between nodes? This version of the problem tends naturally to arise in a wide range of data-intensive science and industry applications whose methodological solution has not appeared in the literature to date; therefore, we feel it is necessary to undertake such analysis. Based on the nonparametric functional statistical viewpoint of large-scale inference, started in Mukhopadhyay (2016), this paper furnishes a new computing model that brings unexpected…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
