Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration

TL;DR

This paper introduces Tilt and Average (TNA), a geometric method for recalibrating neural network classifiers by adjusting the last layer's weights based on angular geometry, improving prediction confidence alignment.

Contribution

We propose a novel geometric adjustment method for the last layer of classifiers, distinct from calibration maps, validated both empirically and theoretically.

Findings

TNA improves calibration performance over baseline methods.

The approach is validated through empirical experiments.

Theoretical analysis supports the effectiveness of TNA.

Abstract

After the revelation that neural networks tend to produce overconfident predictions, the problem of calibration, which aims to align confidence with accuracy to enhance the reliability of predictions, has gained significant importance. Several solutions based on calibration maps have been proposed to address the problem of recalibrating a trained classifier using additional datasets. In this paper, we offer an algorithm that transforms the weights of the last layer of the classifier, distinct from the calibration-map-based approach. We concentrate on the geometry of the final linear layer, specifically its angular aspect, and adjust the weights of the corresponding layer. We name the method Tilt and Average(\textsc{Tna}), and validate the calibration effect empirically and theoretically. Through this, we demonstrate that our approach, in addition to the existing calibration-map-based techniques, can yield improved calibration performance. Code available : https://github.com/GYYYYYUUUUU/TNA_Angular_Scaling.