Coping with Change: Learning Invariant and Minimum Sufficient Representations for Fine-Grained Visual Categorization

TL;DR

This paper introduces a novel approach combining invariant risk minimization and information bottleneck principles to learn invariant and minimal representations for fine-grained visual categorization, improving generalization to unseen data.

Contribution

It proposes a new information-theoretic method for FGVC that addresses distribution shifts and enhances feature robustness and generalization.

Findings

Consistent performance improvements on benchmark datasets.

Effective stabilization of training via matrix-based Rényi entropy.

First to address FGVC from a generalization perspective using IMS.

Abstract

Fine-grained visual categorization (FGVC) is a challenging task due to similar visual appearances between various species. Previous studies always implicitly assume that the training and test data have the same underlying distributions, and that features extracted by modern backbone architectures remain discriminative and generalize well to unseen test data. However, we empirically justify that these conditions are not always true on benchmark datasets. To this end, we combine the merits of invariant risk minimization (IRM) and information bottleneck (IB) principle to learn invariant and minimum sufficient (IMS) representations for FGVC, such that the overall model can always discover the most succinct and consistent fine-grained features. We apply the matrix-based R{\'e}nyi's $\alpha$-order entropy to simplify and stabilize the training of IB; we also design a ``soft" environment partition scheme to make IRM applicable to FGVC task. To the best of our knowledge, we are the first to address the problem of FGVC from a generalization perspective and develop a new information-theoretic solution accordingly. Extensive experiments demonstrate the consistent performance gain offered by our IMS.