Invariant Risk Minimization

TL;DR

Invariant Risk Minimization (IRM) is a new learning approach that aims to find data representations with stable, invariant correlations across different training environments, improving out-of-distribution generalization by capturing causal structures.

Contribution

IRM introduces a novel framework for learning invariant representations that align with causal factors, advancing out-of-distribution robustness in machine learning models.

Findings

IRM learns representations with invariant correlations across distributions.

Theoretical analysis links IRM invariances to causal structures.

Experiments demonstrate IRM's improved out-of-distribution generalization.

Abstract

We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.