PatchNR: Learning from Very Few Images by Patch Normalizing Flow Regularization

TL;DR

PatchNR introduces a novel regularizer using patch-based normalizing flows trained on very few images, enabling high-quality inverse problem solutions across various imaging tasks without extensive data.

Contribution

The paper presents a patch normalizing flow regularizer that is independent of specific inverse problems and effective with extremely limited training images.

Findings

Achieves high-quality results in low-dose and limited-angle CT

Effective superresolution with only a single training image

Combines internal learning for natural image superresolution

Abstract

Learning neural networks using only few available information is an important ongoing research topic with tremendous potential for applications. In this paper, we introduce a powerful regularizer for the variational modeling of inverse problems in imaging. Our regularizer, called patch normalizing flow regularizer (patchNR), involves a normalizing flow learned on small patches of very few images. In particular, the training is independent of the considered inverse problem such that the same regularizer can be applied for different forward operators acting on the same class of images. By investigating the distribution of patches versus those of the whole image class, we prove that our model is indeed a MAP approach. Numerical examples for low-dose and limited-angle computed tomography (CT) as well as superresolution of material images demonstrate that our method provides very high quality results. The training set consists of just six images for CT and one image for superresolution. Finally, we combine our patchNR with ideas from internal learning for performing superresolution of natural images directly from the low-resolution observation without knowledge of any high-resolution image.