MMD-Balls as Credal Sets: A PAC-Bayesian Framework for Epistemic Uncertainty in Test-Time Adaptation

TL;DR

This paper introduces a PAC-Bayesian framework linking distribution shift to prediction reliability using MMD-balls as credal sets, enabling formal epistemic uncertainty quantification in test-time adaptation.

Contribution

It interprets MMD-balls as credal sets within Walley's theory, providing new bounds and uncertainty measures for model adaptation under distribution shift.

Findings

Established a PAC-Bayesian bound with an MMD-dependent shift penalty.

Derived a finite-sample version using MMD concentration.

Provided a worst-case risk bound over credal sets.

Abstract

Test-time adaptation (TTA) methods improve model performance under distribution shift but lack formal guarantees connecting shift magnitude to prediction reliability. We develop a PAC-Bayesian framework yielding generalization bounds explicitly parameterized by the maximum mean discrepancy (MMD) between source and target distributions. Our principal contribution is interpreting MMD-balls around the source distribution as credal sets in Walley's imprecise probability theory, yielding natural epistemic uncertainty quantification. We establish: (i) a PAC-Bayesian bound with an MMD-dependent shift penalty under an RKHS-Lipschitz loss assumption; (ii) a finite-sample version via MMD concentration; (iii) a uniform worst-case risk bound over all distributions in the credal set, with a lower-upper risk decomposition; and (iv) geodesic preservation bounds explaining why kernel-guided adaptation protects local feature geometry. The credal set interpretation separates epistemic from aleatoric uncertainty and provides a principled decision criterion for when adaptation is warranted.