Estimating Random Variables from Random Sparse Observations

TL;DR

This paper develops a theoretical framework linking the distribution of posterior probabilities of discrete variables to fixed points of a density evolution operator, with implications for various applications like coding and detection.

Contribution

It introduces a general relation between the distribution of posterior probabilities and fixed points of a density evolution operator for sparse observations, applicable in large systems.

Findings

Established a relation between posterior distribution and density evolution fixed points.

Proved the relation holds asymptotically in large systems with bounded observation dependence.

Discussed applications to coding, detection, and group testing.

Abstract

Let X_1,...., X_n be a collection of iid discrete random variables, and Y_1,..., Y_m a set of noisy observations of such variables. Assume each observation Y_a to be a random function of some a random subset of the X_i's, and consider the conditional distribution of X_i given the observations, namely \mu_i(x_i)\equiv\prob\{X_i=x_i|Y\} (a posteriori probability). We establish a general relation between the distribution of \mu_i, and the fixed points of the associated density evolution operator. Such relation holds asymptotically in the large system limit, provided the average number of variables an observation depends on is bounded. We discuss the relevance of our result to a number of applications, ranging from sparse graph codes, to multi-user detection, to group testing.