Almost lossless analog signal separation and probabilistic uncertainty relations

TL;DR

This paper introduces an information-theoretic framework for analog signal separation, providing bounds on measurement rates and new probabilistic uncertainty relations that extend compressed sensing concepts to low description complexity signals.

Contribution

It develops a novel probabilistic uncertainty relation for signal separation, generalizes compressed sensing beyond sparsity, and strengthens existing results in analog compression.

Findings

Achievability bounds for measurement rates under measurable and H"older continuous separators.

A matching converse for sources with mixed discrete-continuous distributions.

Introduction of regularized probabilistic uncertainty relations for low complexity signals.

Abstract

We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals, modeled as general random vectors, from the noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verd\'u (2010) on analog compression and encompasses, inter alia, inpainting, declipping, super-resolution, the recovery of signals corrupted by impulse noise, and the separation of (e.g., audio or video) signals into two distinct components. The main results we report are general achievability bounds for the compression rate, i.e., the number of measurements relative to the dimension of the ambient space the signals live in, under either measurability or H\"older continuity imposed on the separator. Furthermore, we find a matching converse for sources of mixed discrete-continuous distribution. For measurable separators our proofs are based on a new probabilistic uncertainty relation which shows that the intersection of generic subspaces with general sets of sufficiently small Minkowski dimension is empty. H\"older continuous separators are dealt with by introducing the concept of regularized probabilistic uncertainty relations. The probabilistic uncertainty relations we develop are inspired by embedding results in dynamical systems theory due to Sauer et al. (1991) and---conceptually---parallel classical Donoho-Stark and Elad-Bruckstein uncertainty principles at the heart of compressed sensing theory. Operationally, the new uncertainty relations take the theory of sparse signal separation beyond traditional sparsity---as measured in terms of the number of non-zero entries---to the more general notion of low description complexity as quantified by Minkowski dimension. Finally, our approach also allows to significantly strengthen key results in Wu and Verd\'u (2010).