An Empirical Study of Invariant Risk Minimization

TL;DR

This paper empirically evaluates Invariant Risk Minimization (IRMv1), demonstrating its effectiveness in learning invariant predictors across varying spurious correlations and extending its application to text classification.

Contribution

It provides the first extensive empirical analysis of IRMv1, revealing its strengths and limitations in different settings and extending its use to text classification tasks.

Findings

IRMv1 performs better with wider variation in spurious correlations.

It learns invariant predictors when the underlying relationship is approximately invariant.

The approach can be extended to text classification scenarios.

Abstract

Invariant risk minimization (IRM) (Arjovsky et al., 2019) is a recently proposed framework designed for learning predictors that are invariant to spurious correlations across different training environments. Yet, despite its theoretical justifications, IRM has not been extensively tested across various settings. In an attempt to gain a better understanding of the framework, we empirically investigate several research questions using IRMv1, which is the first practical algorithm proposed to approximately solve IRM. By extending the ColoredMNIST experiment in different ways, we find that IRMv1 (i) performs better as the spurious correlation varies more widely between training environments, (ii) learns an approximately invariant predictor when the underlying relationship is approximately invariant, and (iii) can be extended to an analogous setting for text classification.