Quantized Proximal Averaging Network for Analysis Sparse Coding

TL;DR

This paper introduces a quantized neural network based on proximal averaging for analysis sparse coding, improving efficiency and accuracy in image reconstruction tasks with minimal performance loss under extreme quantization.

Contribution

It proposes a novel trainable, quantized unfolding network for analysis sparse coding that combines convex and non-convex penalties, enhancing efficiency and reconstruction quality.

Findings

Outperforms state-of-the-art unfolding methods in image recovery

Maintains high accuracy with extreme weight quantization

Demonstrates effectiveness in MRI and compressed image reconstruction

Abstract

We solve the analysis sparse coding problem considering a combination of convex and non-convex sparsity promoting penalties. The multi-penalty formulation results in an iterative algorithm involving proximal-averaging. We then unfold the iterative algorithm into a trainable network that facilitates learning the sparsity prior. We also consider quantization of the network weights. Quantization makes neural networks efficient both in terms of memory and computation during inference, and also renders them compatible for low-precision hardware deployment. Our learning algorithm is based on a variant of the ADAM optimizer in which the quantizer is part of the forward pass and the gradients of the loss function are evaluated corresponding to the quantized weights while doing a book-keeping of the high-precision weights. We demonstrate applications to compressed image recovery and magnetic resonance image reconstruction. The proposed approach offers superior reconstruction accuracy and quality than state-of-the-art unfolding techniques and the performance degradation is minimal even when the weights are subjected to extreme quantization.