Semantics-Guided Representation Learning with Applications to Visual Synthesis

TL;DR

This paper introduces a novel angular triplet-neighbor loss (ATNL) to learn meaningful, semantically-rich latent representations that enable smooth, semantic-aware image interpolation for visual synthesis.

Contribution

It proposes ATNL for guiding latent space learning to encode semantic information and spherical semantic interpolation for improved visual synthesis.

Findings

Effective semantic representation learning demonstrated on MNIST and CMU Multi-PIE datasets.

Enables smooth, semantic-aware image interpolation.

Qualitative and quantitative validation of the method's effectiveness.

Abstract

Learning interpretable and interpolatable latent representations has been an emerging research direction, allowing researchers to understand and utilize the derived latent space for further applications such as visual synthesis or recognition. While most existing approaches derive an interpolatable latent space and induces smooth transition in image appearance, it is still not clear how to observe desirable representations which would contain semantic information of interest. In this paper, we aim to learn meaningful representations and simultaneously perform semantic-oriented and visually-smooth interpolation. To this end, we propose an angular triplet-neighbor loss (ATNL) that enables learning a latent representation whose distribution matches the semantic information of interest. With the latent space guided by ATNL, we further utilize spherical semantic interpolation for generating semantic warping of images, allowing synthesis of desirable visual data. Experiments on MNIST and CMU Multi-PIE datasets qualitatively and quantitatively verify the effectiveness of our method.