A State-Space Approach to Nonstationary Discriminant Analysis

TL;DR

This paper introduces a state-space model-based framework for nonstationary discriminant analysis, improving classification accuracy over traditional methods in drifting data environments by handling temporal distribution changes.

Contribution

It develops a unified, model-based approach for nonstationary discriminant analysis using state-space models, including extensions for nonlinear and non-Gaussian drift scenarios.

Findings

Consistent improvements over stationary LDA, QDA, and SVM.

Robustness to noise, missing data, and class imbalance.

Effective handling of temporal distribution shifts.

Abstract

Classical discriminant analysis assumes identically distributed training data, yet in many applications observations are collected over time and the class-conditional distributions drift. This population drift renders stationary classifiers unreliable. We propose a principled, model-based framework that embeds discriminant analysis within state-space models to obtain nonstationary linear discriminant analysis (NSLDA) and nonstationary quadratic discriminant analysis (NSQDA). For linear-Gaussian dynamics, we adapt Kalman smoothing to handle multiple samples per time step and develop two practical extensions: (i) an expectation-maximization (EM) approach that jointly estimates unknown system parameters, and (ii) a Gaussian mixture model (GMM)-Kalman method that simultaneously recovers unobserved time labels and parameters, a scenario common in practice. To address nonlinear or non-Gaussian drift, we employ particle smoothing to estimate time-varying class centroids, yielding fully nonstationary discriminant rules. Extensive simulations demonstrate consistent improvements over stationary linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and support vector machine (SVM) baselines, with robustness to noise, missing data, and class imbalance. This paper establishes a unified and data-efficient foundation for discriminant analysis under temporal distribution shift.