Accurate and Reliable Predictions with Mutual-Transport Ensemble

TL;DR

The paper introduces the mutual-transport ensemble (MTE), a novel method that co-trains an auxiliary model with adaptive regularization to improve both accuracy and uncertainty calibration in deep neural networks.

Contribution

It proposes the MTE approach that jointly enhances prediction accuracy and uncertainty calibration without sacrificing performance, validated through extensive benchmark studies.

Findings

MTE improves accuracy by up to 3.7% on CIFAR-100.

MTE reduces ECE by up to 42.3%.

MTE outperforms previous state-of-the-art methods.

Abstract

Deep Neural Networks (DNNs) have achieved remarkable success in a variety of tasks, especially when it comes to prediction accuracy. However, in complex real-world scenarios, particularly in safety-critical applications, high accuracy alone is not enough. Reliable uncertainty estimates are crucial. Modern DNNs, often trained with cross-entropy loss, tend to be overconfident, especially with ambiguous samples. To improve uncertainty calibration, many techniques have been developed, but they often compromise prediction accuracy. To tackle this challenge, we propose the ``mutual-transport ensemble'' (MTE). This approach introduces a co-trained auxiliary model and adaptively regularizes the cross-entropy loss using Kullback-Leibler (KL) divergence between the prediction distributions of the primary and auxiliary models. We conducted extensive studies on various benchmarks to validate the effectiveness of our method. The results show that MTE can simultaneously enhance both accuracy and uncertainty calibration. For example, on the CIFAR-100 dataset, our MTE method on ResNet34/50 achieved significant improvements compared to previous state-of-the-art method, with absolute accuracy increases of 2.4%/3.7%, relative reductions in ECE of $42.3%/29.4%, and relative reductions in classwise-ECE of 11.6%/15.3%.