Causal-Driven Feature Evaluation for Cross-Domain Image Classification

TL;DR

This paper proposes a causal evaluation framework for features in cross-domain image classification, improving out-of-distribution generalization by focusing on causal effectiveness rather than invariance.

Contribution

It introduces a segment-level causal evaluation method that directly measures feature necessity and sufficiency across domains, enhancing robustness.

Findings

Consistent OOD performance improvements on multi-domain benchmarks.

Causal evaluation outperforms invariance-based methods under challenging shifts.

Framework provides a more faithful criterion for feature effectiveness.

Abstract

Out-of-distribution (OOD) generalization remains a fundamental challenge in real-world classification, where test distributions often differ substantially from training data. Most existing approaches pursue domain-invariant representations, implicitly assuming that invariance implies reliability. However, features that are invariant across domains are not necessarily causally effective for prediction. In this work, we revisit OOD classification from a causal perspective and propose to evaluate learned representations based on their necessity and sufficiency under distribution shift. We introduce an explicit segment-level framework that directly measures causal effectiveness across domains, providing a more faithful criterion than invariance alone. Experiments on multi-domain benchmarks demonstrate consistent improvements in OOD performance, particularly under challenging domain shifts, highlighting the value of causal evaluation for robust generalization.