Efficient texture-aware multi-GAN for image inpainting

TL;DR

This paper introduces an efficient multi-GAN architecture for image inpainting that enhances texture detail generation and reduces model complexity, enabling high-resolution, realistic inpainting in low-resource settings.

Contribution

The proposed multi-GAN framework improves inpainting quality and efficiency by optimizing four progressive generators and discriminators with a novel LBP-based loss for textures.

Findings

Outperforms state-of-the-art inpainting methods on Places2 and CelebHQ datasets.

Speeds up inference time compared to existing techniques.

Generates high-resolution images with realistic textures.

Abstract

Recent GAN-based (Generative adversarial networks) inpainting methods show remarkable improvements and generate plausible images using multi-stage networks or Contextual Attention Modules (CAM). However, these techniques increase the model complexity limiting their application in low-resource environments. Furthermore, they fail in generating high-resolution images with realistic texture details due to the GAN stability problem. Motivated by these observations, we propose a multi-GAN architecture improving both the performance and rendering efficiency. Our training schema optimizes the parameters of four progressive efficient generators and discriminators in an end-to-end manner. Filling in low-resolution images is less challenging for GANs due to the small dimensional space. Meanwhile, it guides higher resolution generators to learn the global structure consistency of the image. To constrain the inpainting task and ensure fine-grained textures, we adopt an LBP-based loss function to minimize the difference between the generated and the ground truth textures. We conduct our experiments on Places2 and CelebHQ datasets. Qualitative and quantitative results show that the proposed method not only performs favorably against state-of-the-art algorithms but also speeds up the inference time.