Direct and Converse Theorems in Estimating Signals with Sublinear Sparsity

TL;DR

This paper establishes fundamental limits and optimality conditions for estimating signals with sublinear sparsity in noisy environments, validating existing methods and proposing a new non-separable estimator.

Contribution

It provides direct and converse theorems characterizing the performance thresholds of estimators in sublinear sparsity regimes and introduces a heuristic non-separable estimator.

Findings

Maximum likelihood estimator achieves vanishing error below a noise threshold.

Converse results show no estimator can beat a certain error bound above a noise threshold.

The proposed non-separable estimator outperforms Bayesian estimators at high SNR.

Abstract

This paper addresses the estimation of signals with sublinear sparsity sent over the additive white Gaussian noise channel. This fundamental problem arises in designing denoisers used in message-passing algorithms for sublinear sparsity. From a theoretical perspective, the main results are direct and converse theorems in the sublinear sparsity limit, where the signal sparsity grows sublinearly in the signal dimension as the signal dimension tends to infinity. As a direct theorem, the maximum likelihood estimator is proved to achieve vanishing square error in the sublinear sparsity limit if the noise variance is smaller than a threshold. This threshold is known to be achievable by an existing separable Bayesian estimator. As a converse theorem, all estimators cannot achieve square errors smaller than the signal power under a mild condition if the noise variance is larger than another threshold. In particular, the two thresholds coincide with each other when non-zero signals have constant amplitude. These results imply the asymptotic optimality of the existing separable Bayesian estimator used in approximate message-passing for sublinear sparsity. From a numerical perspective, a non-separable estimator is proposed via a heuristic approximation of the true posterior mean estimator. Numerical simulations show that the ML estimator and the proposed non-separable estimator outperform the separable Bayesian estimator for high signal-to-noise ratio (SNR). In the low SNR regime, on the other hand, the two estimators are inferior to the separable Bayesian estimator while the proposed non-separable estimator slightly outperforms the ML estimator.