Transferring disentangled representations: bridging the gap between synthetic and real images

TL;DR

This paper explores transferring disentangled representations from synthetic to real images, demonstrating that some level of effective transfer is possible and proposing a new metric for evaluating disentanglement quality.

Contribution

It introduces an empirical study on transferability of disentangled representations and proposes an interpretable intervention-based metric for assessing factor encoding quality.

Findings

Transfer of disentangled representations from synthetic to real images is feasible.

Some properties of disentanglement are preserved after transfer.

The new metric effectively measures the quality of factor encoding.

Abstract

Developing meaningful and efficient representations that separate the fundamental structure of the data generation mechanism is crucial in representation learning. However, Disentangled Representation Learning has not fully shown its potential on real images, because of correlated generative factors, their resolution and limited access to ground truth labels. Specifically on the latter, we investigate the possibility of leveraging synthetic data to learn general-purpose disentangled representations applicable to real data, discussing the effect of fine-tuning and what properties of disentanglement are preserved after the transfer. We provide an extensive empirical study to address these issues. In addition, we propose a new interpretable intervention-based metric, to measure the quality of factors encoding in the representation. Our results indicate that some level of disentanglement, transferring a representation from synthetic to real data, is possible and effective.