Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing

TL;DR

This paper investigates the identifiability of latent variables in Gaussian mixture models with potential degeneracy, observed through a piecewise affine transformation, and proposes a method leveraging sparsity regularization for recovery.

Contribution

It provides new theoretical identifiability results for degenerate Gaussian mixtures under piecewise affine transformations and introduces a two-stage estimation method.

Findings

Theoretical identifiability results for degenerate Gaussian mixtures.

A two-stage method effectively recovers latent variables in synthetic and image data.

Sparsity regularization enables permutation and scaling identifiability.

Abstract

Causal representation learning (CRL) aims to identify the underlying latent variables from high-dimensional observations, even when variables are dependent with each other. We study this problem for latent variables that follow a potentially degenerate Gaussian mixture distribution and that are only observed through the transformation via a piecewise affine mixing function. We provide a series of progressively stronger identifiability results for this challenging setting in which the probability density functions are ill-defined because of the potential degeneracy. For identifiability up to permutation and scaling, we leverage a sparsity regularization on the learned representation. Based on our theoretical results, we propose a two-stage method to estimate the latent variables by enforcing sparsity and Gaussianity in the learned representations. Experiments on synthetic and image data highlight our method's effectiveness in recovering the ground-truth latent variables.