Calibration of Ordinal Regression Networks

TL;DR

This paper introduces a novel loss function for ordinal regression networks that improves calibration and unimodality of predictions, addressing over-confidence issues inherent in traditional methods.

Contribution

It proposes an ordinal-aware calibration loss with soft ordinal encoding and regularization, achieving state-of-the-art calibration in ordinal regression tasks.

Findings

Achieves state-of-the-art calibration across benchmarks.

Maintains high classification accuracy.

Addresses over-confidence and unimodality issues.

Abstract

Recent studies have shown that deep neural networks are not well-calibrated and often produce over-confident predictions. The miscalibration issue primarily stems from using cross-entropy in classifications, which aims to align predicted softmax probabilities with one-hot labels. In ordinal regression tasks, this problem is compounded by an additional challenge: the expectation that softmax probabilities should exhibit unimodal distribution is not met with cross-entropy. The ordinal regression literature has focused on learning orders and overlooked calibration. To address both issues, we propose a novel loss function that introduces ordinal-aware calibration, ensuring that prediction confidence adheres to ordinal relationships between classes. It incorporates soft ordinal encoding and ordinal-aware regularization to enforce both calibration and unimodality. Extensive experiments across four popular ordinal regression benchmarks demonstrate that our approach achieves state-of-the-art calibration without compromising classification accuracy.