Memory-Augmented Deep Unfolding Network for Compressive Sensing

TL;DR

This paper introduces MADUN, a novel deep unfolding network with memory mechanisms that significantly improves compressive sensing performance by preserving information across stages, inspired by human brain memory processing.

Contribution

The paper proposes a memory-augmented deep unfolding network with HSM and CLM mechanisms, addressing information loss issues in existing DUNs for better CS results.

Findings

MADUN outperforms state-of-the-art methods on natural and MR images.

Memory mechanisms reduce information loss and enhance network representation.

Extensive experiments validate the effectiveness of the proposed approach.

Abstract

Mapping a truncated optimization method into a deep neural network, deep unfolding network (DUN) has attracted growing attention in compressive sensing (CS) due to its good interpretability and high performance. Each stage in DUNs corresponds to one iteration in optimization. By understanding DUNs from the perspective of the human brain's memory processing, we find there exists two issues in existing DUNs. One is the information between every two adjacent stages, which can be regarded as short-term memory, is usually lost seriously. The other is no explicit mechanism to ensure that the previous stages affect the current stage, which means memory is easily forgotten. To solve these issues, in this paper, a novel DUN with persistent memory for CS is proposed, dubbed Memory-Augmented Deep Unfolding Network (MADUN). We design a memory-augmented proximal mapping module (MAPMM) by combining two types of memory augmentation mechanisms, namely High-throughput Short-term Memory (HSM) and Cross-stage Long-term Memory (CLM). HSM is exploited to allow DUNs to transmit multi-channel short-term memory, which greatly reduces information loss between adjacent stages. CLM is utilized to develop the dependency of deep information across cascading stages, which greatly enhances network representation capability. Extensive CS experiments on natural and MR images show that with the strong ability to maintain and balance information our MADUN outperforms existing state-of-the-art methods by a large margin. The source code is available at https://github.com/jianzhangcs/MADUN/.