Evaluating Test-Time Adaptation For Facial Expression Recognition Under Natural Cross-Dataset Distribution Shifts

TL;DR

This paper evaluates the effectiveness of Test-Time Adaptation methods for facial expression recognition under real-world distribution shifts, highlighting how different methods perform depending on the nature and severity of the domain changes.

Contribution

First comprehensive evaluation of TTA methods for FER under natural domain shifts, analyzing performance across various real-world dataset differences.

Findings

TTA can improve FER accuracy by up to 11.34%.

Entropy minimization methods excel with clean target data.

Prototype adjustment methods perform better with larger distributional shifts.

Abstract

Deep learning models often struggle under natural distribution shifts, a common challenge in real-world deployments. Test-Time Adaptation (TTA) addresses this by adapting models during inference without labeled source data. We present the first evaluation of TTA methods for FER under natural domain shifts, performing cross-dataset experiments with widely used FER datasets. This moves beyond synthetic corruptions to examine real-world shifts caused by differing collection protocols, annotation standards, and demographics. Results show TTA can boost FER performance under natural shifts by up to 11.34\%. Entropy minimization methods such as TENT and SAR perform best when the target distribution is clean. In contrast, prototype adjustment methods like T3A excel under larger distributional distance scenarios. Finally, feature alignment methods such as SHOT deliver the largest gains when the target distribution is noisier than our source. Our cross-dataset analysis shows that TTA effectiveness is governed by the distributional distance and the severity of the natural shift across domains.