Domain Generalization: A Survey

TL;DR

This survey comprehensively reviews a decade of research in domain generalization, highlighting methodologies, applications, and future directions for improving out-of-distribution generalization in machine learning.

Contribution

First comprehensive literature review on domain generalization, summarizing developments, methodologies, and future research directions over the past ten years.

Findings

Significant progress in DG methodologies like domain alignment and meta-learning

Diverse applications across vision, speech, NLP, and medical imaging

Identification of key challenges and promising future research areas

Abstract

Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce. This is because most learning algorithms strongly rely on the i.i.d.~assumption on source/target data, which is often violated in practice due to domain shift. Domain generalization (DG) aims to achieve OOD generalization by using only source data for model learning. Over the last ten years, research in DG has made great progress, leading to a broad spectrum of methodologies, e.g., those based on domain alignment, meta-learning, data augmentation, or ensemble learning, to name a few; DG has also been studied in various application areas including computer vision, speech recognition, natural language processing, medical imaging, and reinforcement learning. In this paper, for the first time a comprehensive literature review in DG is provided to summarize the developments over the past decade. Specifically, we first cover the background by formally defining DG and relating it to other relevant fields like domain adaptation and transfer learning. Then, we conduct a thorough review into existing methods and theories. Finally, we conclude this survey with insights and discussions on future research directions.