Test-Time Training with Masked Autoencoders

TL;DR

This paper introduces a test-time training method using masked autoencoders that adapts models to new test distributions on the fly, improving generalization in visual tasks with distribution shifts.

Contribution

It proposes a novel test-time training approach with masked autoencoders, combining empirical improvements with theoretical analysis of bias-variance trade-offs.

Findings

Enhanced generalization on visual benchmarks under distribution shifts

Effective one-sample learning with masked autoencoders

Theoretical insights into bias-variance trade-off improvements

Abstract

Test-time training adapts to a new test distribution on the fly by optimizing a model for each test input using self-supervision. In this paper, we use masked autoencoders for this one-sample learning problem. Empirically, our simple method improves generalization on many visual benchmarks for distribution shifts. Theoretically, we characterize this improvement in terms of the bias-variance trade-off.