On-Demand Learning for Deep Image Restoration

TL;DR

This paper introduces an on-demand learning method for training deep image restoration models that adaptively focus on challenging instances, resulting in improved generalization across various corruption levels.

Contribution

The paper proposes a novel feedback-based on-demand learning algorithm that enhances deep image restoration models' ability to handle diverse corruption levels.

Findings

Outperforms traditional training methods across four restoration tasks

Effective in generalizing to arbitrary corruption levels

Consistent improvements on multiple datasets

Abstract

While machine learning approaches to image restoration offer great promise, current methods risk training models fixated on performing well only for image corruption of a particular level of difficulty---such as a certain level of noise or blur. First, we examine the weakness of conventional "fixated" models and demonstrate that training general models to handle arbitrary levels of corruption is indeed non-trivial. Then, we propose an on-demand learning algorithm for training image restoration models with deep convolutional neural networks. The main idea is to exploit a feedback mechanism to self-generate training instances where they are needed most, thereby learning models that can generalize across difficulty levels. On four restoration tasks---image inpainting, pixel interpolation, image deblurring, and image denoising---and three diverse datasets, our approach consistently outperforms both the status quo training procedure and curriculum learning alternatives.