GEM-FI: Gated Evidential Mixtures with Fisher Modulation

TL;DR

GEM-FI introduces a novel single-pass uncertainty estimation model that improves calibration and out-of-distribution detection by gating evidential outputs with Fisher-informed regularization.

Contribution

It proposes GEM-FI, a new model that stabilizes mixture allocations and captures epistemic multi-modality without multi-pass ensembling.

Findings

GEM-FI improves accuracy on CIFAR-10 from 91.11% to 93.75%.

GEM-FI reduces Brier score significantly, indicating better calibration.

GEM-FI achieves state-of-the-art OOD detection performance on CIFAR benchmarks.

Abstract

Evidential Deep Learning (EDL) enables single-pass uncertainty estimation by predicting Dirichlet evidence, but it can remain overconfident and poorly calibrated, and it often fails to represent multi-modal epistemic uncertainty. We introduce Gated Evidential Mixtures (GEM), a family of models that learns an in-model energy signal and uses it to gate evidential outputs end-to-end in a distance-informed manner. GEM-CORE learns a feature-level energy and maps it to a bounded gate that smoothly suppresses evidence when support is low. To capture epistemic multi-modality without multi-pass ensembling, GEM-MIX adds a lightweight mixture of evidential heads with learned routing weights while preserving single-pass inference. Finally, GEM-FI stabilizes mixture allocations via a Fisher-informed regularizer, reducing head collapse and producing smoother boundary uncertainty. Across image classification and OOD detection benchmarks, GEM improves calibration and ID/OOD separation with single-pass inference. On CIFAR-10, GEM-FI vs. DAEDL improves accuracy from 91.11 to 93.75 (+2.64 pp), reduces Brier x100 from 14.27 to 6.81 (-7.46), and also improves misclassification-detection AUPR from 99.08 to 99.94 (+0.86). For epistemic OOD detection, GEM-FI achieves AUPR/AUROC of 92.59/95.09 on CIFAR-10 to SVHN and 90.20/89.06 on CIFAR-10 to CIFAR-100, compared with 85.54/89.30 and 88.19/86.10 for DAEDL.