ActMAD: Activation Matching to Align Distributions for Test-Time-Training

TL;DR

ActMAD introduces a fine-grained activation matching method for test-time adaptation, aligning feature distributions across multiple layers to improve performance on out-of-distribution data in various tasks.

Contribution

It models activation distributions at multiple layers for more precise adaptation, outperforming existing methods and demonstrating task and architecture independence.

Findings

State-of-the-art results on CIFAR-100C and ImageNet-C

15.4% improvement on KITTI object detection in foggy conditions

Effective online adaptation with minimal data

Abstract

Test-Time-Training (TTT) is an approach to cope with out-of-distribution (OOD) data by adapting a trained model to distribution shifts occurring at test-time. We propose to perform this adaptation via Activation Matching (ActMAD): We analyze activations of the model and align activation statistics of the OOD test data to those of the training data. In contrast to existing methods, which model the distribution of entire channels in the ultimate layer of the feature extractor, we model the distribution of each feature in multiple layers across the network. This results in a more fine-grained supervision and makes ActMAD attain state of the art performance on CIFAR-100C and Imagenet-C. ActMAD is also architecture- and task-agnostic, which lets us go beyond image classification, and score 15.4% improvement over previous approaches when evaluating a KITTI-trained object detector on KITTI-Fog. Our experiments highlight that ActMAD can be applied to online adaptation in realistic scenarios, requiring little data to attain its full performance.