DISPEL: Domain Generalization via Domain-Specific Liberating

TL;DR

DISPEL introduces a flexible post-processing masking method to improve domain generalization by filtering out domain-specific features, leading to better performance on unseen domains across multiple benchmarks.

Contribution

The paper proposes DISPEL, a novel masking approach that effectively filters domain-specific features, enhancing generalization without requiring domain labels or prediction-irrelevant noise removal.

Findings

DISPEL outperforms existing domain generalization methods on five benchmarks.

The method can be applied to various fine-tuned models.

A generalization error bound guarantees performance improvements.

Abstract

Domain generalization aims to learn a generalization model that can perform well on unseen test domains by only training on limited source domains. However, existing domain generalization approaches often bring in prediction-irrelevant noise or require the collection of domain labels. To address these challenges, we consider the domain generalization problem from a different perspective by categorizing underlying feature groups into domain-shared and domain-specific features. Nevertheless, the domain-specific features are difficult to be identified and distinguished from the input data. In this work, we propose DomaIn-SPEcific Liberating (DISPEL), a post-processing fine-grained masking approach that can filter out undefined and indistinguishable domain-specific features in the embedding space. Specifically, DISPEL utilizes a mask generator that produces a unique mask for each input data to filter domain-specific features. The DISPEL framework is highly flexible to be applied to any fine-tuned models. We derive a generalization error bound to guarantee the generalization performance by optimizing a designed objective loss. The experimental results on five benchmarks demonstrate DISPEL outperforms existing methods and can further generalize various algorithms.