Overspecified Mixture Discriminant Analysis: Exponential Convergence, Statistical Guarantees, and Remote Sensing Applications

TL;DR

This paper provides a theoretical analysis of overspecified Mixture Discriminant Analysis, demonstrating exponential convergence of the EM algorithm and statistical guarantees for classification error, with applications to remote sensing data.

Contribution

It offers a rigorous theoretical framework for overspecified MDA, including convergence rates and statistical guarantees, validated through remote sensing experiments.

Findings

EM algorithm converges exponentially fast to Bayes risk with proper initialization.

Classification error converges at a rate of n^{-1/2} under mild conditions.

Experimental validation on remote sensing datasets supports theoretical results.

Abstract

This study explores the classification error of Mixture Discriminant Analysis (MDA) in scenarios where the number of mixture components exceeds those present in the actual data distribution, a condition known as overspecification. We use a two-component Gaussian mixture model within each class to fit data generated from a single Gaussian, analyzing both the algorithmic convergence of the Expectation-Maximization (EM) algorithm and the statistical classification error. We demonstrate that, with suitable initialization, the EM algorithm converges exponentially fast to the Bayes risk at the population level. Further, we extend our results to finite samples, showing that the classification error converges to Bayes risk with a rate $n^{-1/2}$ under mild conditions on the initial parameter estimates and sample size. This work provides a rigorous theoretical framework for understanding the performance of overspecified MDA, which is often used empirically in complex data settings, such as image and text classification. To validate our theory, we conduct experiments on remote sensing datasets.