Normalized Label Distribution: Towards Learning Calibrated, Adaptable and Efficient Activation Maps

TL;DR

This paper introduces normalized soft labels to improve calibration and robustness of neural networks, addressing accuracy, generalizability, and vulnerability to attacks through label normalization and convolutional modifications.

Contribution

It proposes a novel normalized label distribution method that enhances calibration, robustness, and convergence in neural networks, supported by empirical analysis and convolutional adjustments.

Findings

Normalized soft labels improve calibration accuracy.

Label normalization enhances robustness against adversarial attacks.

Convolutional modifications increase convergence speed.

Abstract

The vulnerability of models to data aberrations and adversarial attacks influences their ability to demarcate distinct class boundaries efficiently. The network's confidence and uncertainty play a pivotal role in weight adjustments and the extent of acknowledging such attacks. In this paper, we address the trade-off between the accuracy and calibration potential of a classification network. We study the significance of ground-truth distribution changes on the performance and generalizability of various state-of-the-art networks and compare the proposed method's response to unanticipated attacks. Furthermore, we demonstrate the role of label-smoothing regularization and normalization in yielding better generalizability and calibrated probability distribution by proposing normalized soft labels to enhance the calibration of feature maps. Subsequently, we substantiate our inference by translating conventional convolutions to padding based partial convolution to establish the tangible impact of corrections in reinforcing the performance and convergence rate. We graphically elucidate the implication of such variations with the critical purpose of corroborating the reliability and reproducibility for multiple datasets.