Invariant Meta Learning for Out-of-Distribution Generalization

TL;DR

This paper introduces invariant meta-learning, which enhances out-of-distribution generalization in few-shot learning by finding invariant initializations and applying regularization, outperforming traditional meta-learning methods.

Contribution

It proposes a novel invariant meta-learning approach that explicitly addresses out-of-distribution tasks, improving adaptability and robustness over existing methods.

Findings

Effective on out-of-distribution few-shot tasks

Outperforms traditional meta-learning methods

Demonstrates robustness to distribution shifts

Abstract

Modern deep learning techniques have illustrated their excellent capabilities in many areas, but relies on large training data. Optimization-based meta-learning train a model on a variety tasks, such that it can solve new learning tasks using only a small number of training samples.However, these methods assumes that training and test dataare identically and independently distributed. To overcome such limitation, in this paper, we propose invariant meta learning for out-of-distribution tasks. Specifically, invariant meta learning find invariant optimal meta-initialization,and fast adapt to out-of-distribution tasks with regularization penalty. Extensive experiments demonstrate the effectiveness of our proposed invariant meta learning on out-of-distribution few-shot tasks.