Inferring Sparsity: Compressed Sensing using Generalized Restricted Boltzmann Machines

TL;DR

This paper introduces a Bayesian compressed sensing method that leverages generative models like Restricted Boltzmann Machines, trained on example signals, to improve reconstruction of sparse signals, especially with fewer measurements.

Contribution

It develops a message-passing inference framework for RBMs and integrates it into approximate message passing for enhanced compressed sensing reconstruction.

Findings

Effective reconstruction on MNIST dataset with fewer measurements than sparsity level

Demonstrates the utility of generative models in compressed sensing

Improves reconstruction accuracy over traditional methods

Abstract

In this work, we consider compressed sensing reconstruction from $M$ measurements of $K$-sparse structured signals which do not possess a writable correlation model. Assuming that a generative statistical model, such as a Boltzmann machine, can be trained in an unsupervised manner on example signals, we demonstrate how this signal model can be used within a Bayesian framework of signal reconstruction. By deriving a message-passing inference for general distribution restricted Boltzmann machines, we are able to integrate these inferred signal models into approximate message passing for compressed sensing reconstruction. Finally, we show for the MNIST dataset that this approach can be very effective, even for $M < K$.