Covariance-corrected Whitening Alleviates Network Degeneration on Imbalanced Classification

TL;DR

This paper introduces Whitening-Net, a framework that uses covariance-corrected whitening to address network degeneration caused by class imbalance in image classification, improving model stability and performance.

Contribution

The paper proposes covariance-corrected modules, GRBS and BET, to stabilize whitening in imbalanced data scenarios, enhancing deep recognition models.

Findings

Effective in imbalanced datasets like CIFAR-LT, ImageNet-LT, and iNaturalist-LT.

Improves convergence and stability of whitening in extreme class imbalance.

Enhances recognition accuracy with minimal computational overhead.

Abstract

Class imbalance is a critical issue in image classification that significantly affects the performance of deep recognition models. In this work, we first identify a network degeneration dilemma that hinders the model learning by introducing a high linear dependence among the features inputted into the classifier. To overcome this challenge, we propose a novel framework called Whitening-Net to mitigate the degenerate solutions, in which ZCA whitening is integrated before the linear classifier to normalize and decorrelate the batch samples. However, in scenarios with extreme class imbalance, the batch covariance statistic exhibits significant fluctuations, impeding the convergence of the whitening operation. Therefore, we propose two covariance-corrected modules, the Group-based Relatively Balanced Batch Sampler (GRBS) and the Batch Embedded Training (BET), to get more accurate and stable batch covariance, thereby reinforcing the capability of whitening. Our modules can be trained end-to-end without incurring substantial computational costs. Comprehensive empirical evaluations conducted on benchmark datasets, including CIFAR-LT-10/100, ImageNet-LT, and iNaturalist-LT, validate the effectiveness of our proposed approaches.