Test-Time Modification: Inverse Domain Transformation for Robust Perception

TL;DR

This paper introduces a test-time inverse domain transformation method using diffusion models to improve robustness of perception models across different environments without extensive synthetic data generation.

Contribution

It proposes a novel test-time approach leveraging diffusion models to map target images to source domain, enhancing domain generalization without synthetic data.

Findings

Significant performance improvements across multiple tasks.

Effective in real-to-real domain shifts.

Robustness enhanced with ensemble methods.

Abstract

Generative foundation models contain broad visual knowledge and can produce diverse image variations, making them particularly promising for advancing domain generalization tasks. They can be used for training data augmentation, but synthesizing comprehensive target-domain variations remains slow, expensive, and incomplete. We propose an alternative: using diffusion models at test time to map target images back to the source distribution where the downstream model was trained. This approach requires only a source domain description, preserves the task model, and eliminates large-scale synthetic data generation. We demonstrate consistent improvements across segmentation, detection, and classification tasks under challenging environmental shifts in real-to-real domain generalization scenarios with unknown target distributions. Our analysis spans multiple generative and downstream models, including an ensemble variant for enhanced robustness. The method improves BDD100K-Night-Det mAP@50 from 10.2 to 31.8, ImageNet-R top-1 from 36.1 to 60.8, and DarkZurich mIoU from 28.6 to 46.3.