Impression Space from Deep Template Network

TL;DR

This paper introduces the Impression Space, a novel feature space derived from a pretrained CNN that captures salient image features and enables tasks like image translation and synthesis without additional training.

Contribution

The work proposes a framework to create an Impression Space from a pretrained network, allowing image reconstruction, translation, and relation modeling without retraining.

Findings

Impression Space captures salient features of images.

It enables unpaired image translation and synthesis.

It reveals feature similarities between images.

Abstract

It is an innate ability for humans to imagine something only according to their impression, without having to memorize all the details of what they have seen. In this work, we would like to demonstrate that a trained convolutional neural network also has the capability to "remember" its input images. To achieve this, we propose a simple but powerful framework to establish an {\emph{Impression Space}} upon an off-the-shelf pretrained network. This network is referred to as the {\emph{Template Network}} because its filters will be used as templates to reconstruct images from the impression. In our framework, the impression space and image space are bridged by a layer-wise encoding and iterative decoding process. It turns out that the impression space indeed captures the salient features from images, and it can be directly applied to tasks such as unpaired image translation and image synthesis through impression matching without further network training. Furthermore, the impression naturally constructs a high-level common space for different data. Based on this, we propose a mechanism to model the data relations inside the impression space, which is able to reveal the feature similarity between images. Our code will be released.