Deep Exponential Families

TL;DR

Deep exponential families (DEFs) are a flexible class of hierarchical latent variable models inspired by deep neural networks, capable of capturing complex dependencies and improving predictive performance across various data types.

Contribution

This paper introduces DEFs, a new class of hierarchical latent variable models that extend exponential families and demonstrate improved prediction and data exploration capabilities.

Findings

Going beyond one layer enhances DEFs' predictive accuracy.

DEFs outperform state-of-the-art models in experiments.

DEFs uncover meaningful structures in large datasets.

Abstract

We describe \textit{deep exponential families} (DEFs), a class of latent variable models that are inspired by the hidden structures used in deep neural networks. DEFs capture a hierarchy of dependencies between latent variables, and are easily generalized to many settings through exponential families. We perform inference using recent "black box" variational inference techniques. We then evaluate various DEFs on text and combine multiple DEFs into a model for pairwise recommendation data. In an extensive study, we show that going beyond one layer improves predictions for DEFs. We demonstrate that DEFs find interesting exploratory structure in large data sets, and give better predictive performance than state-of-the-art models.