Improving Detail in Pluralistic Image Inpainting with Feature Dequantization

TL;DR

This paper introduces a Feature Dequantization Module (FDM) to improve detail quality in VQGAN-based pluralistic image inpainting, effectively compensating for information loss caused by feature quantization.

Contribution

The paper proposes a novel FDM that restores image details in VQGAN-based inpainting and presents an efficient training method reducing costs.

Findings

FDM significantly improves detail quality in generated images.

The method achieves this with negligible additional training and inference costs.

Empirical results demonstrate enhanced structural integrity and visual quality.

Abstract

Pluralistic Image Inpainting (PII) offers multiple plausible solutions for restoring missing parts of images and has been successfully applied to various applications including image editing and object removal. Recently, VQGAN-based methods have been proposed and have shown that they significantly improve the structural integrity in the generated images. Nevertheless, the state-of-the-art VQGAN-based model PUT faces a critical challenge: degradation of detail quality in output images due to feature quantization. Feature quantization restricts the latent space and causes information loss, which negatively affects the detail quality essential for image inpainting. To tackle the problem, we propose the FDM (Feature Dequantization Module) specifically designed to restore the detail quality of images by compensating for the information loss. Furthermore, we develop an efficient training method for FDM which drastically reduces training costs. We empirically demonstrate that our method significantly enhances the detail quality of the generated images with negligible training and inference overheads.