Domain Generalization through Meta-Learning: A Survey

TL;DR

This survey reviews how meta-learning techniques can enhance domain generalization in deep neural networks, addressing challenges posed by out-of-distribution data and domain shifts in real-world applications.

Contribution

It introduces a new taxonomy of meta-learning methods for domain generalization and provides a decision graph to guide model selection based on data and domain shift characteristics.

Findings

Comprehensive mapping of meta-learning approaches for domain generalization.

Identification of key challenges and promising research directions.

Practical guidance for selecting suitable meta-learning models.

Abstract

Deep neural networks (DNNs) have revolutionized artificial intelligence but often lack performance when faced with out-of-distribution (OOD) data, a common scenario due to the inevitable domain shifts in real-world applications. This limitation stems from the common assumption that training and testing data share the same distribution--an assumption frequently violated in practice. Despite their effectiveness with large amounts of data and computational power, DNNs struggle with distributional shifts and limited labeled data, leading to overfitting and poor generalization across various tasks and domains. Meta-learning presents a promising approach by employing algorithms that acquire transferable knowledge across various tasks for fast adaptation, eliminating the need to learn each task from scratch. This survey paper delves into the realm of meta-learning with a focus on its contribution to domain generalization. We first clarify the concept of meta-learning for domain generalization and introduce a novel taxonomy based on the feature extraction strategy and the classifier learning methodology, offering a granular view of methodologies. Additionally, we present a decision graph to assist readers in navigating the taxonomy based on data availability and domain shifts, enabling them to select and develop a proper model tailored to their specific problem requirements. Through an exhaustive review of existing methods and underlying theories, we map out the fundamentals of the field. Our survey provides practical insights and an informed discussion on promising research directions.