Mix and Reason: Reasoning over Semantic Topology with Data Mixing for Domain Generalization

TL;DR

This paper introduces Mix and Reason (ire), a novel domain generalization framework that enforces semantic topology invariance through data mixing and topology refinement, improving generalization to unseen domains.

Contribution

The paper proposes a new DG method combining data mixing and semantic topology refinement, addressing limitations of disentanglement assumptions in real-world scenarios.

Findings

ire outperforms existing DG methods on multiple benchmarks.

It demonstrates robustness and effectiveness in unseen domain generalization.

The approach maintains semantic invariance across diverse domains.

Abstract

Domain generalization (DG) enables generalizing a learning machine from multiple seen source domains to an unseen target one. The general objective of DG methods is to learn semantic representations that are independent of domain labels, which is theoretically sound but empirically challenged due to the complex mixture of common and domain-specific factors. Although disentangling the representations into two disjoint parts has been gaining momentum in DG, the strong presumption over the data limits its efficacy in many real-world scenarios. In this paper, we propose Mix and Reason (\mire), a new DG framework that learns semantic representations via enforcing the structural invariance of semantic topology. \mire\ consists of two key components, namely, Category-aware Data Mixing (CDM) and Adaptive Semantic Topology Refinement (ASTR). CDM mixes two images from different domains in virtue of activation maps generated by two complementary classification losses, making the classifier focus on the representations of semantic objects. ASTR introduces relation graphs to represent semantic topology, which is progressively refined via the interactions between local feature aggregation and global cross-domain relational reasoning. Experiments on multiple DG benchmarks validate the effectiveness and robustness of the proposed \mire.