Riemannian optimization for non-centered mixture of scaled Gaussian distributions

TL;DR

This paper introduces a Riemannian gradient descent algorithm for non-centered mixtures of scaled Gaussian distributions, enabling efficient minimization of likelihood and divergence problems with applications in classification.

Contribution

It develops a novel Riemannian optimization framework for NC-MSGs, including algorithms for likelihood minimization and center of mass computation, with theoretical guarantees and practical applications.

Findings

The algorithm performs well and is fast on the tested problems.

The KL divergence between NC-MSGs is explicitly derived.

The classifier achieves good accuracy and robustness on large-scale data.

Abstract

This paper studies the statistical model of the non-centered mixture of scaled Gaussian distributions (NC-MSG). Using the Fisher-Rao information geometry associated to this distribution, we derive a Riemannian gradient descent algorithm. This algorithm is leveraged for two minimization problems. The first one is the minimization of a regularized negative log-likelihood (NLL). The latter makes the trade-off between a white Gaussian distribution and the NC-MSG. Conditions on the regularization are given so that the existence of a minimum to this problem is guaranteed without assumptions on the samples. Then, the Kullback-Leibler (KL) divergence between two NC-MSG is derived. This divergence enables us to define a minimization problem to compute centers of mass of several NC-MSGs. The proposed Riemannian gradient descent algorithm is leveraged to solve this second minimization problem. Numerical experiments show the good performance and the speed of the Riemannian gradient descent on the two problems. Finally, a Nearest centroid classifier is implemented leveraging the KL divergence and its associated center of mass. Applied on the large scale dataset Breizhcrops, this classifier shows good accuracies as well as robustness to rigid transformations of the test set.