Addressing the Topological Defects of Disentanglement via Distributed Operators

TL;DR

This paper investigates the limitations of traditional disentanglement methods that assign factors to separate subspaces, revealing topological defects, and proposes a distributed operator approach that better handles affine transformations in data.

Contribution

It introduces a novel distributed operator framework for disentanglement, overcoming topological defects of previous subspace methods, supported by theoretical and empirical evidence.

Findings

Traditional methods cause discontinuities in encoding affine transformations.

Distributed operators effectively disentangle affine transformations.

The approach provides a theoretical foundation for recent models using distributed operators.

Abstract

A core challenge in Machine Learning is to learn to disentangle natural factors of variation in data (e.g. object shape vs. pose). A popular approach to disentanglement consists in learning to map each of these factors to distinct subspaces of a model's latent representation. However, this approach has shown limited empirical success to date. Here, we show that, for a broad family of transformations acting on images--encompassing simple affine transformations such as rotations and translations--this approach to disentanglement introduces topological defects (i.e. discontinuities in the encoder). Motivated by classical results from group representation theory, we study an alternative, more flexible approach to disentanglement which relies on distributed latent operators, potentially acting on the entire latent space. We theoretically and empirically demonstrate the effectiveness of this approach to disentangle affine transformations. Our work lays a theoretical foundation for the recent success of a new generation of models using distributed operators for disentanglement.