Deeply Aggregated Alternating Minimization for Image Restoration

TL;DR

This paper introduces DeepAM, a novel deep learning framework that enhances traditional alternating minimization for image restoration by learning key steps with neural networks, leading to improved performance across various tasks.

Contribution

It presents a general deep learning-based framework that replaces parts of the AM algorithm with neural networks trained end-to-end, improving image restoration results.

Findings

Outperforms recent data-driven and nonlocal methods.

Effective in tasks like denoising, RGB-NIR restoration, and depth super-resolution.

Demonstrates flexibility and improved accuracy across multiple image restoration tasks.

Abstract

Regularization-based image restoration has remained an active research topic in computer vision and image processing. It often leverages a guidance signal captured in different fields as an additional cue. In this work, we present a general framework for image restoration, called deeply aggregated alternating minimization (DeepAM). We propose to train deep neural network to advance two of the steps in the conventional AM algorithm: proximal mapping and ?- continuation. Both steps are learned from a large dataset in an end-to-end manner. The proposed framework enables the convolutional neural networks (CNNs) to operate as a prior or regularizer in the AM algorithm. We show that our learned regularizer via deep aggregation outperforms the recent data-driven approaches as well as the nonlocalbased methods. The flexibility and effectiveness of our framework are demonstrated in several image restoration tasks, including single image denoising, RGB-NIR restoration, and depth super-resolution.