Bayesian-based Online Label Shift Estimation with Dynamic Dirichlet Priors

TL;DR

This paper introduces Bayesian methods with dynamic Dirichlet priors for online label shift estimation, significantly improving test prior accuracy and classifier performance in streaming and imbalanced data scenarios.

Contribution

It proposes FMAPLS and online-FMAPLS, novel Bayesian algorithms with closed-form hyperparameter updates for real-time label shift estimation.

Findings

Achieves up to 40% lower KL divergence on CIFAR100 and ImageNet.

Improves post-shift classification accuracy under severe class imbalance.

Demonstrates robustness and scalability in large-scale, dynamic environments.

Abstract

Label shift, a prevalent challenge in supervised learning, arises when the class prior distribution of test data differs from that of training data, leading to significant degradation in classifier performance. To accurately estimate the test priors and enhance classification accuracy, we propose a Bayesian framework for label shift estimation, termed Full Maximum A Posterior Label Shift (FMAPLS), along with its online version, online-FMAPLS. Leveraging batch and online Expectation-Maximization (EM) algorithms, these methods jointly and dynamically optimize Dirichlet hyperparameters $\boldsymbol{\alpha}$ and class priors $\boldsymbol{\pi}$, thereby overcoming the rigid constraints of the existing Maximum A Posterior Label Shift (MAPLS) approach. Moreover, we introduce a linear surrogate function (LSF) to replace gradient-based hyperparameter updates, yielding closed-form solutions that reduce computational complexity while retaining asymptotic equivalence. The online variant substitutes the batch E-step with a stochastic approximation, enabling real-time adaptation to streaming data. Furthermore, our theoretical analysis reveals a fundamental trade-off between online convergence rate and estimation accuracy. Extensive experiments on CIFAR100 and ImageNet datasets under shuffled long-tail and Dirichlet test priors demonstrate that FMAPLS and online-FMAPLS respectively achieve up to 40% and 12% lower KL divergence and substantial improvements in post-shift accuracy over state-of-the-art baselines, particularly under severe class imbalance and distributional uncertainty. These results confirm the robustness, scalability, and suitability of the proposed methods for large-scale and dynamic learning scenarios.