Learning Sparse Mixture Models

TL;DR

This paper introduces a method for learning high-dimensional sparse mixture models using an ANOVA-like structure with wrapped Gaussian and von Mises distributions, reducing complexity and improving accuracy.

Contribution

It proposes an algorithm to identify active variables and their interactions, enabling efficient learning of sparse mixture models in high dimensions.

Findings

Reduces computational complexity for high-dimensional data.

Improves model accuracy with sample-based learning.

Demonstrates effectiveness through numerical examples.

Abstract

This work approximates high-dimensional density functions with an ANOVA-like sparse structure by the mixture of wrapped Gaussian and von Mises distributions. When the dimension $d$ is very large, it is complex and impossible to train the model parameters by the usually known learning algorithms due to the curse of dimensionality. Therefore, assuming that each component of the model depends on an a priori unknown much smaller number of variables than the space dimension $d,$ we first define an algorithm that determines the mixture model's set of active variables by the Kolmogorov-Smirnov and correlation test. Then restricting the learning procedure to the set of active variables, we iteratively determine the set of variable interactions of the marginal density function and simultaneously learn the parameters by the Kolmogorov and correlation coefficient statistic test and the proximal Expectation-Maximization algorithm. The learning procedure considerably reduces the algorithm's complexity for the input dimension $d$ and increases the model's accuracy for the given samples, as the numerical examples show.