Efficient Approximate Recovery from Pooled Data Using Doubly Regular Pooling Schemes

TL;DR

This paper introduces a doubly regular pooling scheme and an approximate greedy algorithm for efficiently recovering hidden binary states in pooled data, tolerating some errors and noise, with proven performance guarantees.

Contribution

It presents a novel doubly regular test design and an analysis of an approximate recovery algorithm that is robust to noise and sparsity, advancing pooled data reconstruction methods.

Findings

Algorithm achieves low error probability in noisy settings

Performance analysis is uniform across noise levels and sparsity

Simulations confirm effectiveness for realistic sample sizes

Abstract

In the pooled data problem we are given $n$ agents with hidden state bits, either $0$ or $1$. The hidden states are unknown and can be seen as the underlying ground truth $\sigma$. To uncover that ground truth, we are given a querying method that queries multiple agents at a time. Each query reports the sum of the states of the queried agents. Our goal is to learn the hidden state bits using as few queries as possible. So far, most literature deals with exact reconstruction of all hidden state bits. We study a more relaxed variant in which we allow a small fraction of agents to be classified incorrectly. This becomes particularly relevant in the noisy variant of the pooled data problem where the queries' results are subject to random noise. In this setting, we provide a doubly regular test design that assigns agents to queries. For this design we analyze an approximate reconstruction algorithm that estimates the hidden bits in a greedy fashion. We give a rigorous analysis of the algorithm's performance, its error probability, and its approximation quality. As a main technical novelty, our analysis is uniform in the degree of noise and the sparsity of $\sigma$. Finally, simulations back up our theoretical findings and provide strong empirical evidence that our algorithm works well for realistic sample sizes.