V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network

TL;DR

V-LinkNet introduces a novel dual encoder architecture with recursive residual transition layers and a contextualized feature loss to improve high-quality image inpainting, especially in deep feature layers, demonstrating superior results on multiple datasets.

Contribution

The paper proposes a new dual encoder inpainting model with recursive residual transition layers and a contextualized feature loss, enhancing deep feature transfer and inpainting quality.

Findings

V-LinkNet outperforms existing models on CelebA-HQ and Paris Street View datasets.

The recursive residual transition layer effectively increases semantic information transfer.

Standardized protocol reduces bias in mask-image pairing during evaluation.

Abstract

Image inpainting is a key technique in image processing task to predict the missing regions and generate realistic images. Given the advancement of existing generative inpainting models with feature extraction, propagation and reconstruction capabilities, there is lack of high-quality feature extraction and transfer mechanisms in deeper layers to tackle persistent aberrations on the generated inpainted regions. Our method, V-LinkNet, develops high-level feature transference to deep level textural context of inpainted regions our work, proposes a novel technique of combining encoders learning through a recursive residual transition layer (RSTL). The RSTL layer easily adapts dual encoders by increasing the unique semantic information through direct communication. By collaborating the dual encoders structure with contextualised feature representation loss function, our system gains the ability to inpaint with high-level features. To reduce biases from random mask-image pairing, we introduce a standard protocol with paired mask-image on the testing set of CelebA-HQ, Paris Street View and Places2 datasets. Our results show V-LinkNet performed better on CelebA-HQ and Paris Street View using this standard protocol. We will share the standard protocol and our codes with the research community upon acceptance of this paper.