Energy Aligning for Biased Models

TL;DR

This paper introduces Energy Aligning, a simple method to reduce bias in models trained on class-imbalanced data by adjusting output logits, improving performance in class incremental learning and long-tailed recognition tasks.

Contribution

The paper proposes a novel, architecture-agnostic energy aligning technique that effectively mitigates class bias without altering training procedures or network structures.

Findings

Energy aligning reduces class bias in imbalanced datasets.

The method outperforms state-of-the-art approaches on benchmarks.

It is simple, effective, and applicable to various tasks.

Abstract

Training on class-imbalanced data usually results in biased models that tend to predict samples into the majority classes, which is a common and notorious problem. From the perspective of energy-based model, we demonstrate that the free energies of categories are aligned with the label distribution theoretically, thus the energies of different classes are expected to be close to each other when aiming for ``balanced'' performance. However, we discover a severe energy-bias phenomenon in the models trained on class-imbalanced dataset. To eliminate the bias, we propose a simple and effective method named Energy Aligning by merely adding the calculated shift scalars onto the output logits during inference, which does not require to (i) modify the network architectures, (ii) intervene the standard learning paradigm, (iii) perform two-stage training. The proposed algorithm is evaluated on two class imbalance-related tasks under various settings: class incremental learning and long-tailed recognition. Experimental results show that energy aligning can effectively alleviate class imbalance issue and outperform state-of-the-art methods on several benchmarks.