Relevance Factor VAE: Learning and Identifying Disentangled Factors

TL;DR

Relevance Factor VAE is an unsupervised deep auto-encoder that learns to identify and disentangle meaningful sources of variation in data, improving interpretability and performance over existing methods.

Contribution

The paper introduces Relevance-Factor-VAE, which uses relevance indicators and focuses total correlation loss on meaningful factors, advancing disentanglement in unsupervised learning.

Findings

Outperforms existing methods on benchmark datasets

Effectively distinguishes meaningful from nuisance factors

Introduces a new disentanglement metric

Abstract

We propose a novel VAE-based deep auto-encoder model that can learn disentangled latent representations in a fully unsupervised manner, endowed with the ability to identify all meaningful sources of variation and their cardinality. Our model, dubbed Relevance-Factor-VAE, leverages the total correlation (TC) in the latent space to achieve the disentanglement goal, but also addresses the key issue of existing approaches which cannot distinguish between meaningful and nuisance factors of latent variation, often the source of considerable degradation in disentanglement performance. We tackle this issue by introducing the so-called relevance indicator variables that can be automatically learned from data, together with the VAE parameters. Our model effectively focuses the TC loss onto the relevant factors only by tolerating large prior KL divergences, a desideratum justified by our semi-parametric theoretical analysis. Using a suite of disentanglement metrics, including a newly proposed one, as well as qualitative evidence, we demonstrate that our model outperforms existing methods across several challenging benchmark datasets.