Mixture Representation Learning with Coupled Autoencoders

TL;DR

This paper introduces cpl-mixVAE, an unsupervised variational autoencoder framework that effectively learns mixture representations involving high-dimensional discrete and continuous factors, demonstrated on gene expression data.

Contribution

The paper presents a scalable variational framework with multiple interacting networks and a consensus constraint for disentangling complex mixture factors without supervision.

Findings

Successfully uncovers discrete and continuous factors in gene expression data.

Scales well to high-dimensional discrete latent spaces.

Provides theoretical and experimental validation of the approach.

Abstract

Jointly identifying a mixture of discrete and continuous factors of variability without supervision is a key problem in unraveling complex phenomena. Variational inference has emerged as a promising method to learn interpretable mixture representations. However, posterior approximation in high-dimensional latent spaces, particularly for discrete factors remains challenging. Here, we propose an unsupervised variational framework using multiple interacting networks called cpl-mixVAE that scales well to high-dimensional discrete settings. In this framework, the mixture representation of each network is regularized by imposing a consensus constraint on the discrete factor. We justify the use of this framework by providing both theoretical and experimental results. Finally, we use the proposed method to jointly uncover discrete and continuous factors of variability describing gene expression in a single-cell transcriptomic dataset profiling more than a hundred cortical neuron types.