Revisiting Essential and Nonessential Settings of Evidential Deep Learning

TL;DR

This paper revisits Evidential Deep Learning (EDL), identifies nonessential components, and proposes Re-EDL, a simplified variant that improves uncertainty estimation by relaxing certain settings and optimizing core components.

Contribution

The paper introduces Re-EDL, a simplified and more effective version of EDL, by relaxing nonessential settings and focusing on essential aspects like the projected probability.

Findings

Re-EDL achieves state-of-the-art performance on uncertainty estimation tasks.

Relaxing nonessential settings improves model calibration and predictive uncertainty.

Extensive experiments validate the effectiveness of Re-EDL.

Abstract

Evidential Deep Learning (EDL) is an emerging method for uncertainty estimation that provides reliable predictive uncertainty in a single forward pass, attracting significant attention. Grounded in subjective logic, EDL derives Dirichlet concentration parameters from neural networks to construct a Dirichlet probability density function (PDF), modeling the distribution of class probabilities. Despite its success, EDL incorporates several nonessential settings: In model construction, (1) a commonly ignored prior weight parameter is fixed to the number of classes, while its value actually impacts the balance between the proportion of evidence and its magnitude in deriving predictive scores. In model optimization, (2) the empirical risk features a variance-minimizing optimization term that biases the PDF towards a Dirac delta function, potentially exacerbating overconfidence. (3) Additionally, the structural risk typically includes a KL-divergence-minimizing regularization, whose optimization direction extends beyond the intended purpose and contradicts common sense, diminishing the information carried by the evidence magnitude. Therefore, we propose Re-EDL, a simplified yet more effective variant of EDL, by relaxing the nonessential settings and retaining the essential one, namely, the adoption of projected probability from subjective logic. Specifically, Re-EDL treats the prior weight as an adjustable hyperparameter rather than a fixed scalar, and directly optimizes the expectation of the Dirichlet PDF provided by deprecating both the variance-minimizing optimization term and the divergence regularization term. Extensive experiments and state-of-the-art performance validate the effectiveness of our method. The source code is available at https://github.com/MengyuanChen21/Re-EDL.