Learning a Sketch Tensor Space for Image Inpainting of Man-made Scenes

TL;DR

This paper introduces a novel Sketch Tensor space and a multi-scale inpainting network to effectively restore structural details like edges and junctions in man-made scene images, improving inpainting quality.

Contribution

It proposes a new Sketch Tensor space and a multi-scale encoder-decoder network specifically designed for structure-preserving inpainting of man-made scenes.

Findings

Effective restoration of edges, lines, and junctions in man-made scenes.

Competitive performance on general natural image inpainting.

Validated through extensive experiments.

Abstract

This paper studies the task of inpainting man-made scenes. It is very challenging due to the difficulty in preserving the visual patterns of images, such as edges, lines, and junctions. Especially, most previous works are failed to restore the object/building structures for images of man-made scenes. To this end, this paper proposes learning a Sketch Tensor (ST) space for inpainting man-made scenes. Such a space is learned to restore the edges, lines, and junctions in images, and thus makes reliable predictions of the holistic image structures. To facilitate the structure refinement, we propose a Multi-scale Sketch Tensor inpainting (MST) network, with a novel encoder-decoder structure. The encoder extracts lines and edges from the input images to project them into an ST space. From this space, the decoder is learned to restore the input images. Extensive experiments validate the efficacy of our model. Furthermore, our model can also achieve competitive performance in inpainting general nature images over the competitors.