Image Restoration with Locally Selected Class-Adapted Models

TL;DR

This paper introduces a method for image restoration that adaptively selects class-specific models for different regions within an image, improving results by combining segmentation and restoration.

Contribution

It extends previous class-adapted denoising techniques to handle images with multiple classes and regions, enabling local model selection during restoration.

Findings

Outperforms general-purpose algorithms on class-specific images

Effectively handles images with multiple regions of different classes

Simultaneously performs segmentation and restoration

Abstract

State-of-the-art algorithms for imaging inverse problems (namely deblurring and reconstruction) are typically iterative, involving a denoising operation as one of its steps. Using a state-of-the-art denoising method in this context is not trivial, and is the focus of current work. Recently, we have proposed to use a class-adapted denoiser (patch-based using Gaussian mixture models) in a so-called plug-and-play scheme, wherein a state-of-the-art denoiser is plugged into an iterative algorithm, leading to results that outperform the best general-purpose algorithms, when applied to an image of a known class (e.g. faces, text, brain MRI). In this paper, we extend that approach to handle situations where the image being processed is from one of a collection of possible classes or, more importantly, contains regions of different classes. More specifically, we propose a method to locally select one of a set of class-adapted Gaussian mixture patch priors, previously estimated from clean images of those classes. Our approach may be seen as simultaneously performing segmentation and restoration, thus contributing to bridging the gap between image restoration/reconstruction and analysis.