Learning Exponential Family Graphical Models with Latent Variables using Regularized Conditional Likelihood

TL;DR

This paper introduces a convex relaxation method based on regularized conditional likelihood for learning exponential family graphical models with latent variables, addressing confounding dependencies without requiring explicit latent variable distribution.

Contribution

It proposes a novel, broadly applicable convex relaxation framework for latent-variable graphical modeling that does not depend on the distribution of latent variables.

Findings

Effective on synthetic data

Applicable to real-world datasets

Handles non-Gaussian data

Abstract

Fitting a graphical model to a collection of random variables given sample observations is a challenging task if the observed variables are influenced by latent variables, which can induce significant confounding statistical dependencies among the observed variables. We present a new convex relaxation framework based on regularized conditional likelihood for latent-variable graphical modeling in which the conditional distribution of the observed variables conditioned on the latent variables is given by an exponential family graphical model. In comparison to previously proposed tractable methods that proceed by characterizing the marginal distribution of the observed variables, our approach is applicable in a broader range of settings as it does not require knowledge about the specific form of distribution of the latent variables and it can be specialized to yield tractable approaches to problems in which the observed data are not well-modeled as Gaussian. We demonstrate the utility and flexibility of our framework via a series of numerical experiments on synthetic as well as real data.