Cascade context encoder for improved inpainting

TL;DR

This paper proposes a cascade context encoder approach for image inpainting that improves plausibility by sequentially increasing input size, and introduces a new quantitative measure based on latent feature properties.

Contribution

It introduces a novel cascade method for context encoders and formalizes a new inpainting evaluation metric based on latent features.

Findings

Visibly more plausible inpainting results with the cascade approach.

First formalization of inpainting quality measure based on latent feature properties.

Provides a new quantitative metric for inpainting evaluation.

Abstract

In this paper, we analyze if cascade usage of the context encoder with increasing input can improve the results of the inpainting. For this purpose, we train context encoder for 64x64 pixels images in a standard way and use its resized output to fill in the missing input region of the 128x128 context encoder, both in training and evaluation phase. As the result, the inpainting is visibly more plausible. In order to thoroughly verify the results, we introduce normalized squared-distortion, a measure for quantitative inpainting evaluation, and we provide its mathematical explanation. This is the first attempt to formalize the inpainting measure, which is based on the properties of latent feature representation, instead of L2 reconstruction loss.