Deep Spatial and Tonal Data Optimisation for Homogeneous Diffusion Inpainting

TL;DR

This paper introduces a neural network-based method for rapid optimization of data points in homogeneous diffusion inpainting, enabling real-time high-quality image reconstruction for applications like image compression.

Contribution

It presents the first neural network architecture that efficiently optimizes pixel positions and values for diffusion inpainting, surpassing traditional model-based methods in speed.

Findings

Achieves over 10,000x faster data optimization compared to traditional methods.

Enables real-time high-quality image inpainting.

Uses a novel surrogate solver with backpropagation for diffusion inpainting.

Abstract

Diffusion-based inpainting can reconstruct missing image areas with high quality from sparse data, provided that their location and their values are well optimised. This is particularly useful for applications such as image compression, where the original image is known. Selecting the known data constitutes a challenging optimisation problem, that has so far been only investigated with model-based approaches. So far, these methods require a choice between either high quality or high speed since qualitatively convincing algorithms rely on many time-consuming inpaintings. We propose the first neural network architecture that allows fast optimisation of pixel positions and pixel values for homogeneous diffusion inpainting. During training, we combine two optimisation networks with a neural network-based surrogate solver for diffusion inpainting. This novel concept allows us to perform backpropagation based on inpainting results that approximate the solution of the inpainting equation. Without the need for a single inpainting during test time, our deep optimisation accelerates data selection by more than four orders of magnitude compared to common model-based approaches. This provides real-time performance with high quality results.