Unsupervised Disentanglement with Tensor Product Representations on the Torus

TL;DR

This paper introduces a novel auto-encoder approach that employs tensor product representations on a torus to achieve naturally disentangled latent spaces, outperforming traditional methods in unsupervised disentanglement tasks.

Contribution

The work proposes using tensor product structures on a torus for latent representations, providing a new way to achieve disentanglement in auto-encoders.

Findings

Improved disentanglement, completeness, and informativeness over existing methods.

Latent space distributed uniformly over unit circles, capturing generative factors effectively.

Demonstrated advantages through extensive experiments.

Abstract

The current methods for learning representations with auto-encoders almost exclusively employ vectors as the latent representations. In this work, we propose to employ a tensor product structure for this purpose. This way, the obtained representations are naturally disentangled. In contrast to the conventional variations methods, which are targeted toward normally distributed features, the latent space in our representation is distributed uniformly over a set of unit circles. We argue that the torus structure of the latent space captures the generative factors effectively. We employ recent tools for measuring unsupervised disentanglement, and in an extensive set of experiments demonstrate the advantage of our method in terms of disentanglement, completeness, and informativeness. The code for our proposed method is available at https://github.com/rotmanmi/Unsupervised-Disentanglement-Torus.