TIDE: Training Locally Interpretable Domain Generalization Models Enables Test-time Correction

TL;DR

TIDE introduces a novel training scheme and test-time correction method for domain generalization that emphasizes local concepts and provides interpretability, significantly improving robustness across diverse datasets.

Contribution

The paper proposes a new training approach with concept saliency and contrastive losses, along with a test-time correction algorithm leveraging local concept representations.

Findings

Achieves 12% average improvement over state-of-the-art on four benchmarks.

Provides interpretable predictions via concept saliency maps.

Demonstrates robustness to semantic shifts like background and viewpoint changes.

Abstract

We consider the problem of single-source domain generalization. Existing methods typically rely on extensive augmentations to synthetically cover diverse domains during training. However, they struggle with semantic shifts (e.g., background and viewpoint changes), as they often learn global features instead of local concepts that tend to be domain invariant. To address this gap, we propose an approach that compels models to leverage such local concepts during prediction. Given no suitable dataset with per-class concepts and localization maps exists, we first develop a novel pipeline to generate annotations by exploiting the rich features of diffusion and large-language models. Our next innovation is TIDE, a novel training scheme with a concept saliency alignment loss that ensures model focus on the right per-concept regions and a local concept contrastive loss that promotes learning domain-invariant concept representations. This not only gives a robust model but also can be visually interpreted using the predicted concept saliency maps. Given these maps at test time, our final contribution is a new correction algorithm that uses the corresponding local concept representations to iteratively refine the prediction until it aligns with prototypical concept representations that we store at the end of model training. We evaluate our approach extensively on four standard DG benchmark datasets and substantially outperform the current state-ofthe-art (12% improvement on average) while also demonstrating that our predictions can be visually interpreted