Domain Generalization for Domain-Linked Classes
Kimathi Kaai, Saad Hossain, Sirisha Rambhatla
TL;DR
This paper introduces a new approach called FOND to improve domain generalization for classes that are only expressed in specific domains, by transferring knowledge from domain-shared classes, achieving state-of-the-art results.
Contribution
The paper defines the task of domain-linked class generalization and proposes the FOND algorithm to address it, demonstrating its effectiveness through extensive experiments.
Findings
FOND outperforms baselines on popular DG benchmarks for domain-linked classes.
Transferring knowledge from domain-shared classes improves generalization for domain-linked classes.
Practical insights on data conditions enhance real-world applicability.
Abstract
Domain generalization (DG) focuses on transferring domain-invariant knowledge from multiple source domains (available at train time) to an, a priori, unseen target domain(s). This requires a class to be expressed in multiple domains for the learning algorithm to break the spurious correlations between domain and class. However, in the real-world, classes may often be domain-linked, i.e. expressed only in a specific domain, which leads to extremely poor generalization performance for these classes. In this work, we aim to learn generalizable representations for these domain-linked classes by transferring domain-invariant knowledge from classes expressed in multiple source domains (domain-shared classes). To this end, we introduce this task to the community and propose a Fair and cONtrastive feature-space regularization algorithm for Domain-linked DG, FOND. Rigorous and reproducible…
Peer Reviews
Decision·Submitted to ICLR 2024
Indeed, modeling that explicitly considers the relationship between domains and classes is not extensively developed in existing methodologies. In this regard, addressing this specific aspect presents a novel approach to problem-solving in the field. This innovative focus could provide significant advancements in understanding and tackling domain-specific challenges. It's reasonable to assume that domain-linked classes might have limited data compared to domain-shared classes. If the informatio
The simplicity of the proposed methodology, which essentially relies on contrastive learning based on domain-shared classes and aligns the losses between domain-linked and domain-shared classes, does seem straightforward. While leveraging information from domain-shared classes to inform domain-linked classes could be beneficial, it's understandable to question whether such loss matching alone suffices to supply rich information. Furthermore, the connection between merely aligning loss magnitude
- This paper introduces a new setting of domain generalization where classes can be domain-shared or domain-linked. - The proposed method applies fairness for the domain-linked classes.
(1) In section 5.2, the description of fairness is somewhat unclear. Is $M$ referring to the model, specifically the neural network? If so, it seems that the fairness loss is intended to reduce the classification loss gap between domain-linked and domain-shared classes, suggesting that minimizing the fairness loss aims to make the classification loss for both types of classes have similar values during training. However, it would be helpful to clarify how exactly this loss relates to fairness.
Novelty: The paper addresses a less-explored area in DG — the challenge posed by domain-linked classes, which significantly hinders the performance of generalization models. Quality: The introduction of the FOND algorithm, which aims to improve the generalizability of domain-linked classes by utilizing domain-shared class representations, is a noteworthy methodological contribution.
1. Significance: The practical applicability of the research is questionable, given that the empirical validation is conducted on synthetic datasets, which may not effectively simulate real-world complexities. 2. Quality: The theoretical analysis lacks depth, presenting generalized bounds without significant divergence from existing domain generalization theories, thereby offering limited novel insights. 3. Novelty: The paper's innovation is constrained, primarily adapting existing fairness me
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications