Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification

TL;DR

This paper introduces xERM, a method that trains unbiased long-tailed classifiers by balancing domain risks, improving performance across different test distributions, especially when test data is also long-tailed.

Contribution

The paper proposes xERM, a novel approach that addresses bias in long-tailed classification by balancing risks across domains, supported by theoretical causality analysis.

Findings

xERM outperforms existing methods on long-tailed benchmarks.

It achieves balanced performance on both head and tail classes.

Theoretical analysis explains unbiasedness via domain risk adjustment.

Abstract

We address the overlooked unbiasedness in existing long-tailed classification methods: we find that their overall improvement is mostly attributed to the biased preference of tail over head, as the test distribution is assumed to be balanced; however, when the test is as imbalanced as the long-tailed training data -- let the test respect Zipf's law of nature -- the tail bias is no longer beneficial overall because it hurts the head majorities. In this paper, we propose Cross-Domain Empirical Risk Minimization (xERM) for training an unbiased model to achieve strong performances on both test distributions, which empirically demonstrates that xERM fundamentally improves the classification by learning better feature representation rather than the head vs. tail game. Based on causality, we further theoretically explain why xERM achieves unbiasedness: the bias caused by the domain selection is removed by adjusting the empirical risks on the imbalanced domain and the balanced but unseen domain. Codes are available at https://github.com/BeierZhu/xERM.