Identifying Metric Structures of Deep Latent Variable Models

TL;DR

This paper proposes a method to identify meaningful relationships like distances and angles between latent variables in deep models, improving interpretability without requiring labeled data.

Contribution

It introduces a novel approach to analyze relationships between latent variables, bypassing the need for identifiability of individual latent factors.

Findings

More reliable latent distances are achieved.

Theoretical conditions for relationship identification are established.

Empirical results demonstrate improved interpretability.

Abstract

Deep latent variable models learn condensed representations of data that, hopefully, reflect the inner workings of the studied phenomena. Unfortunately, these latent representations are not statistically identifiable, meaning they cannot be uniquely determined. Domain experts, therefore, need to tread carefully when interpreting these. Current solutions limit the lack of identifiability through additional constraints on the latent variable model, e.g. by requiring labeled training data, or by restricting the expressivity of the model. We change the goal: instead of identifying the latent variables, we identify relationships between them such as meaningful distances, angles, and volumes. We prove this is feasible under very mild model conditions and without additional labeled data. We empirically demonstrate that our theory results in more reliable latent distances, offering a principled path forward in extracting trustworthy conclusions from deep latent variable models.