A Rigorous Link Between Self-Organizing Maps and Gaussian Mixture Models

TL;DR

This paper establishes a rigorous mathematical connection between Self-Organizing Maps and Gaussian Mixture Models, showing that SOMs can be interpreted as performing gradient descent on an approximation of GMM likelihood, thus providing a probabilistic perspective.

Contribution

It provides a formal mathematical link between SOMs and GMMs, interpreting SOMs as gradient descent on a GMM likelihood approximation, which was not previously established.

Findings

SOMs can be viewed as gradient descent on GMM likelihoods.

The neighborhood radius decrease in SOMs acts as an annealing process.

This link enables SOMs to be used as probabilistic generative models.

Abstract

This work presents a mathematical treatment of the relation between Self-Organizing Maps (SOMs) and Gaussian Mixture Models (GMMs). We show that energy-based SOM models can be interpreted as performing gradient descent, minimizing an approximation to the GMM log-likelihood that is particularly valid for high data dimensionalities. The SOM-like decrease of the neighborhood radius can be understood as an annealing procedure ensuring that gradient descent does not get stuck in undesirable local minima. This link allows to treat SOMs as generative probabilistic models, giving a formal justification for using SOMs, e.g., to detect outliers, or for sampling.