Memory augment is All You Need for image restoration

TL;DR

MemoryNet introduces a novel memory layer and contrastive learning approach for image restoration, effectively enhancing performance across various degradation tasks with improved perceptual quality.

Contribution

The paper proposes MemoryNet, a new model with a three-granularity memory layer and contrastive learning, addressing transparency issues and improving restoration results.

Findings

Significant PSNR and SSIM gains on three datasets.

Effective in derain, deshadow, and deblur tasks.

Produces perceptually realistic restored images.

Abstract

Image restoration is a low-level vision task, most CNN methods are designed as a black box, lacking transparency and internal aesthetics. Although some methods combining traditional optimization algorithms with DNNs have been proposed, they all have some limitations. In this paper, we propose a three-granularity memory layer and contrast learning named MemoryNet, specifically, dividing the samples into positive, negative, and actual three samples for contrastive learning, where the memory layer is able to preserve the deep features of the image and the contrastive learning converges the learned features to balance. Experiments on Derain/Deshadow/Deblur task demonstrate that these methods are effective in improving restoration performance. In addition, this paper's model obtains significant PSNR, SSIM gain on three datasets with different degradation types, which is a strong proof that the recovered images are perceptually realistic. The source code of MemoryNet can be obtained from https://github.com/zhangbaijin/MemoryNet