Optimal Clustering in Anisotropic Gaussian Mixture Models

TL;DR

This paper investigates optimal clustering under anisotropic Gaussian Mixture Models with unknown, non-identical covariances, establishing theoretical bounds and proposing an efficient algorithm that achieves optimal rates.

Contribution

It provides the first minimax lower bounds for clustering with anisotropic covariances and introduces a practical hard EM algorithm that attains these bounds.

Findings

Derived minimax lower bounds for clustering accuracy.

Proposed a hard EM algorithm that converges to the optimal rate.

Algorithm is computationally feasible and effective in anisotropic settings.

Abstract

We study the clustering task under anisotropic Gaussian Mixture Models where the covariance matrices from different clusters are unknown and are not necessarily the identical matrix. We characterize the dependence of signal-to-noise ratios on the cluster centers and covariance matrices and obtain the minimax lower bound for the clustering problem. In addition, we propose a computationally feasible procedure and prove it achieves the optimal rate within a few iterations. The proposed procedure is a hard EM type algorithm, and it can also be seen as a variant of the Lloyd's algorithm that is adjusted to the anisotropic covariance matrices.