Smart, Sparse Contours to Represent and Edit Images

TL;DR

This paper introduces a GAN-based method for reconstructing high-fidelity images from sparse contours, enabling intuitive semantic editing and achieving results indistinguishable from original images.

Contribution

The authors present a novel contour-based image reconstruction technique that requires less than 6% of pixels, allowing for semantic editing and high-quality synthesis.

Findings

Reconstruction quality comparable to source images.

Sparse contours suffice for detailed image synthesis.

Semantic edits produce coherent long-range changes.

Abstract

We study the problem of reconstructing an image from information stored at contour locations. We show that high-quality reconstructions with high fidelity to the source image can be obtained from sparse input, e.g., comprising less than $6\%$ of image pixels. This is a significant improvement over existing contour-based reconstruction methods that require much denser input to capture subtle texture information and to ensure image quality. Our model, based on generative adversarial networks, synthesizes texture and details in regions where no input information is provided. The semantic knowledge encoded into our model and the sparsity of the input allows to use contours as an intuitive interface for semantically-aware image manipulation: local edits in contour domain translate to long-range and coherent changes in pixel space. We can perform complex structural changes such as changing facial expression by simple edits of contours. Our experiments demonstrate that humans as well as a face recognition system mostly cannot distinguish between our reconstructions and the source images.