The interplay of signal-to-noise ratio and variance misspecification in Gaussian mixtures

TL;DR

This paper analyzes how variance misspecification affects estimation and clustering in Gaussian mixture models, revealing phase transitions and the impact of SNR on bias and cluster recovery.

Contribution

It characterizes the effects of variance misspecification on Gaussian mixture models, deriving phase diagrams and asymptotic behaviors under different SNR regimes.

Findings

Under correct specification, MLE recovers true means regardless of SNR.

Variance misspecification causes mean displacement or cluster collapse depending on smoothing.

Low SNR leads to near-random clustering, with bias from misspecification and hard assignments.

Abstract

We study estimation and clustering in Gaussian mixture models under variance misspecification. Observations are generated with true variance $\sigma^2$, while the component means are estimated using a likelihood with variance $\tau^2$, yielding a family of mismatched likelihood functions parameterized by the ratio $\rho=\tau/\sigma$. We show that the interplay between $\rho$ and the signal-to-noise ratio (SNR) induces a sharp phase diagram. Under correct specification ($\rho=1$), maximum likelihood recovers the true means, independently of the SNR. However, once the model is misspecified, two different regimes emerge. Under under-smoothing ($\rho<1$), the estimated Gaussian means are displaced from the truth, and in low SNR this discrepancy grows as the SNR decreases: for every fixed $\rho<1$, the squared error scales as $\mathrm{SNR}^{-1}$. Under over-smoothing ($\rho>1$), the fitted likelihood blurs the cluster separation, causing distinct component means to collapse towards the overall mixture center once $\rho^2$ exceeds a threshold of the form $1 + \lambda\,\mathrm{SNR}$, where $\lambda$ depends on the geometry of the true means. We further show that the hard assignment objective arises as the limit $\tau\to 0$ of the same mismatched likelihood family, and derive corresponding low- and high-SNR results for hard-assignment mean estimation and latent-label recovery. Furthermore, in low SNR, Bayes-optimal clustering is close to random guessing, and the hard-assignment target remains far from the true means. These results show that in low-SNR applications, even mild variance misspecification or hard-assignment procedures can induce substantial bias, whereas in high SNR these effects are largely absent.