A latent-observed dissimilarity measure

TL;DR

This paper introduces a new dissimilarity measure called latent-observed dissimilarity (LOD) to evaluate the relationship between latent and observed variables in generative models, aiding model comparison and understanding.

Contribution

The paper proposes the LOD measure and defines four types of generative models, demonstrating its effectiveness in capturing model differences and the impact of conditional independence on information transmission.

Findings

LOD effectively captures differences between models

Conditional independence improves information flow to higher layers

LOD reflects the capability for higher layer learning

Abstract

Quantitatively assessing relationships between latent variables and observed variables is important for understanding and developing generative models and representation learning. In this paper, we propose latent-observed dissimilarity (LOD) to evaluate the dissimilarity between the probabilistic characteristics of latent and observed variables. We also define four essential types of generative models with different independence/conditional independence configurations. Experiments using tractable real-world data show that LOD can effectively capture the differences between models and reflect the capability for higher layer learning. They also show that the conditional independence of latent variables given observed variables contributes to improving the transmission of information and characteristics from lower layers to higher layers.