RandNet: deep learning with compressed measurements of images

TL;DR

RandNet is a neural network framework that learns representations directly from compressed random measurements of images, enabling efficient training and maintaining high accuracy even with significant data compression.

Contribution

It introduces RandNet, a novel neural network architecture capable of learning from compressed measurements, reducing memory usage and computational cost during training.

Findings

RandNet successfully performs dictionary learning from compressed data.

It classifies MNIST images with minimal accuracy loss using 50% compressed measurements.

Training efficiency improves with sparse measurements due to reduced computation.

Abstract

Principal component analysis, dictionary learning, and auto-encoders are all unsupervised methods for learning representations from a large amount of training data. In all these methods, the higher the dimensions of the input data, the longer it takes to learn. We introduce a class of neural networks, termed RandNet, for learning representations using compressed random measurements of data of interest, such as images. RandNet extends the convolutional recurrent sparse auto-encoder architecture to dense networks and, more importantly, to the case when the input data are compressed random measurements of the original data. Compressing the input data makes it possible to fit a larger number of batches in memory during training. Moreover, in the case of sparse measurements,training is more efficient computationally. We demonstrate that, in unsupervised settings, RandNet performs dictionary learning using compressed data. In supervised settings, we show that RandNet can classify MNIST images with minimal loss in accuracy, despite being trained with random projections of the images that result in a 50% reduction in size. Overall, our results provide a general principled framework for training neural networks using compressed data.