DiffGANPaint: Fast Inpainting Using Denoising Diffusion GANs

TL;DR

DiffGANPaint introduces a fast inpainting method combining diffusion models with GANs, enabling efficient reconstruction of missing image parts with improved performance over existing methods.

Contribution

The paper proposes a novel DDPM-based inpainting model that leverages GAN generators to accelerate sampling, enhancing efficiency and generalization to unseen mask types.

Findings

Performs on par or better than current state-of-the-art methods

Achieves faster inpainting sampling times

Generalizes well to various mask types

Abstract

Free-form image inpainting is the task of reconstructing parts of an image specified by an arbitrary binary mask. In this task, it is typically desired to generalize model capabilities to unseen mask types, rather than learning certain mask distributions. Capitalizing on the advances in diffusion models, in this paper, we propose a Denoising Diffusion Probabilistic Model (DDPM) based model capable of filling missing pixels fast as it models the backward diffusion process using the generator of a generative adversarial network (GAN) network to reduce sampling cost in diffusion models. Experiments on general-purpose image inpainting datasets verify that our approach performs superior or on par with most contemporary works.