Latent Regression Bayesian Network for Data Representation

TL;DR

This paper introduces a novel latent regression Bayesian network that models dependencies among latent variables using a pseudo-likelihood inference method and employs a hard EM algorithm for learning, enhancing data representation and reconstruction.

Contribution

It proposes a new inference approach preserving latent variable dependencies and utilizes hard EM for efficient learning in deep directed generative models.

Findings

Effective data representation demonstrated on benchmark datasets

Improved reconstruction quality over state-of-the-art models

Preservation of latent variable dependencies enhances model fidelity

Abstract

Deep directed generative models have attracted much attention recently due to their expressive representation power and the ability of ancestral sampling. One major difficulty of learning directed models with many latent variables is the intractable inference. To address this problem, most existing algorithms make assumptions to render the latent variables independent of each other, either by designing specific priors, or by approximating the true posterior using a factorized distribution. We believe the correlations among latent variables are crucial for faithful data representation. Driven by this idea, we propose an inference method based on the conditional pseudo-likelihood that preserves the dependencies among the latent variables. For learning, we propose to employ the hard Expectation Maximization (EM) algorithm, which avoids the intractability of the traditional EM by max-out instead of sum-out to compute the data likelihood. Qualitative and quantitative evaluations of our model against state of the art deep models on benchmark datasets demonstrate the effectiveness of the proposed algorithm in data representation and reconstruction.