Disentanglement with Factor Quantized Variational Autoencoders

TL;DR

This paper introduces FactorQVAE, a discrete variational autoencoder that learns disentangled representations without ground truth factors, using scalar quantization and total correlation to improve disentanglement and reconstruction.

Contribution

The work proposes a novel discrete VAE model with scalar quantization and an inductive bias to enhance disentanglement without requiring known generative factors.

Findings

Outperforms existing disentanglement methods on DCI and InfoMEC metrics

Improves reconstruction quality compared to prior approaches

Demonstrates the effectiveness of discrete representations in disentanglement

Abstract

Disentangled representation learning aims to represent the underlying generative factors of a dataset in a latent representation independently of one another. In our work, we propose a discrete variational autoencoder (VAE) based model where the ground truth information about the generative factors are not provided to the model. We demonstrate the advantages of learning discrete representations over learning continuous representations in facilitating disentanglement. Furthermore, we propose incorporating an inductive bias into the model to further enhance disentanglement. Precisely, we propose scalar quantization of the latent variables in a latent representation with scalar values from a global codebook, and we add a total correlation term to the optimization as an inductive bias. Our method called FactorQVAE combines optimization based disentanglement approaches with discrete representation learning, and it outperforms the former disentanglement methods in terms of two disentanglement metrics (DCI and InfoMEC) while improving the reconstruction performance. Our code can be found at https://github.com/ituvisionlab/FactorQVAE.