Improvement of Batch Normalization in Imbalanced Data

TL;DR

This paper proposes a simple modification to batch normalization to address size-mismatch issues when combining it with weighted loss functions in neural network classification tasks on imbalanced datasets, improving performance.

Contribution

It introduces a novel modification to batch normalization that effectively handles data imbalance when used with weighted loss functions in neural networks.

Findings

Modified BN improves classification accuracy on imbalanced datasets

Combining weighted loss with modified BN outperforms standard methods

The approach is simple and easy to implement

Abstract

In this study, we consider classification problems based on neural networks in data-imbalanced environment. Learning from an imbalanced data set is one of the most important and practical problems in the field of machine learning. A weighted loss function based on cost-sensitive approach is a well-known effective method for imbalanced data sets. We consider a combination of weighted loss function and batch normalization (BN) in this study. BN is a powerful standard technique in the recent developments in deep learning. A simple combination of both methods leads to a size-mismatch problem due to a mismatch between interpretations of effective size of data set in both methods. We propose a simple modification to BN to correct the size-mismatch and demonstrate that this modified BN is effective in data-imbalanced environment.