ASAP: Unsupervised Post-training with Label Distribution Shift Adaptive Learning Rate

TL;DR

ASAP introduces a lightweight, unsupervised method for adapting machine learning models to changing label distributions by dynamically adjusting the learning rate based on output shifts, without needing labels or past inputs.

Contribution

The paper presents ASAP, a novel unsupervised post-training approach that adaptively tunes the learning rate using cosine distance, enabling effective online adaptation to label shift.

Findings

ASAP improves accuracy across multiple datasets.

ASAP is computationally efficient and requires no labels.

ASAP effectively handles various label shift scenarios.

Abstract

In real-world applications, machine learning models face online label shift, where label distributions change over time. Effective adaptation requires careful learning rate selection: too low slows adaptation and too high causes instability. We propose ASAP (Adaptive Shift Aware Post-training), which dynamically adjusts the learning rate by computing the cosine distance between current and previous unlabeled outputs and mapping it within a bounded range. ASAP requires no labels, model ensembles, or past inputs, using only the previous softmax output for fast, lightweight adaptation. Experiments across multiple datasets and shift scenarios show ASAP consistently improves accuracy and efficiency, making it practical for unsupervised model adaptation.