A Tutorial on Deep Latent Variable Models of Natural Language

TL;DR

This tutorial explains how deep latent variable models combine probabilistic modeling with deep learning, addressing inference challenges using variational inference.

Contribution

It provides a comprehensive overview of deep latent variable models, highlighting recent advances and discussing solutions to inference difficulties.

Findings

Deep latent variable models enable flexible probabilistic modeling with deep neural networks.

Variational inference is a key technique to address intractability in these models.

The tutorial clarifies how to implement and optimize deep latent variable models.

Abstract

There has been much recent, exciting work on combining the complementary strengths of latent variable models and deep learning. Latent variable modeling makes it easy to explicitly specify model constraints through conditional independence properties, while deep learning makes it possible to parameterize these conditional likelihoods with powerful function approximators. While these "deep latent variable" models provide a rich, flexible framework for modeling many real-world phenomena, difficulties exist: deep parameterizations of conditional likelihoods usually make posterior inference intractable, and latent variable objectives often complicate backpropagation by introducing points of non-differentiability. This tutorial explores these issues in depth through the lens of variational inference.