Context is All You Need

TL;DR

CONTXT is a lightweight, easy-to-implement method that enhances neural network robustness under domain shifts by using simple feature transforms for contextual adaptation during test time.

Contribution

It introduces CONTXT, a novel, simple method for neural feature modulation that improves domain generalization and test-time adaptation with minimal overhead.

Findings

CONTXT improves performance across classification and generative tasks.

The method is lightweight and easy to integrate into existing models.

CONTXT yields consistent gains under domain shift scenarios.

Abstract

Artificial Neural Networks (ANNs) are increasingly deployed across diverse real-world settings, where they must operate under data distributions that differ from those seen during training. This challenge is central to Domain Generalization (DG), which trains models to generalize to unseen domains without target data, and Test-Time Adaptation (TTA), which improves robustness by adapting to unlabeled test data at deployment. Existing approaches to address these challenges are often complex, resource-intensive, and difficult to scale. We introduce CONTXT (Contextual augmentatiOn for Neural feaTure X Transforms), a simple and intuitive method for contextual adaptation. CONTXT modulates internal representations using simple additive and multiplicative feature transforms. Within a TTA setting, it yields consistent gains across discriminative tasks (e.g., ANN/CNN classification) and generative models (e.g., LLMs). The method is lightweight, easy to integrate, and incurs minimal overhead, enabling robust performance under domain shift without added complexity. More broadly, CONTXT provides a compact way to steer information flow and neural processing without retraining.