Deep Gaussian Mixture Models

TL;DR

Deep Gaussian Mixture Models extend traditional mixture models into a hierarchical deep learning framework, enabling flexible, nonlinear data modeling through layered Gaussian mixtures with potential dimension reduction.

Contribution

Introduction of Deep Gaussian Mixture Models with layered Gaussian mixtures and the integration of factor models for dimension reduction.

Findings

Deep GMMs can model complex data relationships.

Layered structure enhances modeling flexibility.

Dimension reduction prevents overparameterization.

Abstract

Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, Deep Gaussian Mixture Models are introduced and discussed. A Deep Gaussian Mixture model (DGMM) is a network of multiple layers of latent variables, where, at each layer, the variables follow a mixture of Gaussian distributions. Thus, the deep mixture model consists of a set of nested mixtures of linear models, which globally provide a nonlinear model able to describe the data in a very flexible way. In order to avoid overparameterized solutions, dimension reduction by factor models can be applied at each layer of the architecture thus resulting in deep mixtures of factor analysers.