Exploring the Efficacy of Group-Normalization in Deep Learning Models for Alzheimer's Disease Classification

TL;DR

This paper investigates Group Normalization as an effective alternative to Batch Normalization in deep learning models, especially for small batch sizes, demonstrating comparable or improved accuracy in Alzheimer's disease classification tasks.

Contribution

It introduces Group Normalization as a batch-size-independent normalization method that outperforms Batch Normalization in small-batch scenarios and is easily transferable between training phases.

Findings

GN achieves 10.6% error rate with batch size 2 on ImageNet.

GN performs comparably to BN on standard batch sizes.

GN outperforms other normalization techniques in small-batch training.

Abstract

Batch Normalization is an important approach to advancing deep learning since it allows multiple networks to train simultaneously. A problem arises when normalizing along the batch dimension because B.N.'s error increases significantly as batch size shrinks because batch statistics estimates are inaccurate. As a result, computer vision tasks like detection, segmentation, and video, which require tiny batches based on memory consumption, aren't suitable for using Batch Normalization for larger model training and feature transfer. Here, we explore Group Normalization as an easy alternative to using Batch Normalization A Group Normalization is a channel normalization method in which each group is divided into different channels, and the corresponding mean and variance are calculated for each group. Group Normalization computations are accurate across a wide range of batch sizes and are independent of batch size. When trained using a large ImageNet database on ResNet-50, GN achieves a very low error rate of 10.6% compared to Batch Normalization. when a smaller batch size of only 2 is used. For usual batch sizes, the performance of G.N. is comparable to that of Batch Normalization, but at the same time, it outperforms other normalization techniques. Implementing Group Normalization as a direct alternative to B.N to combat the serious challenges faced by the Batch Normalization in deep learning models with comparable or improved classification accuracy. Additionally, Group Normalization can be naturally transferred from the pre-training to the fine-tuning phase. .