Composing graphical models with neural networks for structured representations and fast inference

TL;DR

This paper introduces a unified framework that integrates probabilistic graphical models with neural networks, enabling structured representations and efficient inference for complex data, demonstrated through various models and a behavioral study.

Contribution

It presents a novel scalable approach combining graphical models and deep learning, extending variational autoencoders with graphical structure and recognition networks.

Findings

Framework effectively combines graphical models with neural networks.

Scalable inference achieved via stochastic variational methods.

Application demonstrates utility in behavioral phenotyping.

Abstract

We propose a general modeling and inference framework that composes probabilistic graphical models with deep learning methods and combines their respective strengths. Our model family augments graphical structure in latent variables with neural network observation models. For inference, we extend variational autoencoders to use graphical model approximating distributions with recognition networks that output conjugate potentials. All components of these models are learned simultaneously with a single objective, giving a scalable algorithm that leverages stochastic variational inference, natural gradients, graphical model message passing, and the reparameterization trick. We illustrate this framework with several example models and an application to mouse behavioral phenotyping.