Deep Feature Response Discriminative Calibration

TL;DR

This paper introduces a discriminative feature response calibration method for deep neural networks, enhancing feature discriminability and model performance by integrating Gaussian-based confidence values into a modified ResNet architecture.

Contribution

The paper proposes a novel discriminative calibration technique for neural feature responses and incorporates it into ResNet, improving model accuracy across multiple datasets.

Findings

Improved accuracy on CIFAR-10, CIFAR-100, SVHN, and ImageNet.

Neural feature responses follow a Gaussian distribution.

Effective plugin-based calibration module enhances feature discriminability.

Abstract

Deep neural networks (DNNs) have numerous applications across various domains. Several optimization techniques, such as ResNet and SENet, have been proposed to improve model accuracy. These techniques improve the model performance by adjusting or calibrating feature responses according to a uniform standard. However, they lack the discriminative calibration for different features, thereby introducing limitations in the model output. Therefore, we propose a method that discriminatively calibrates feature responses. The preliminary experimental results indicate that the neural feature response follows a Gaussian distribution. Consequently, we compute confidence values by employing the Gaussian probability density function, and then integrate these values with the original response values. The objective of this integration is to improve the feature discriminability of the neural feature response. Based on the calibration values, we propose a plugin-based calibration module incorporated into a modified ResNet architecture, termed Response Calibration Networks (ResCNet). Extensive experiments on datasets like CIFAR-10, CIFAR-100, SVHN, and ImageNet demonstrate the effectiveness of the proposed approach. The developed code is publicly available at https://github.com/tcmyxc/ResCNet.