Learn to Accumulate Evidence from All Training Samples: Theory and Practice
Deep Pandey, Qi Yu

TL;DR
This paper investigates the limitations of evidential deep learning models, identifies the cause of their inferior performance, and proposes a new regularizer to improve their effectiveness on large-scale datasets.
Contribution
The paper provides a theoretical analysis of evidential models, uncovers a fundamental limitation related to zero evidence regions, and introduces a novel regularizer to address this issue.
Findings
Theoretical analysis explains the cause of poor performance in evidential models.
Proposed regularizer alleviates the zero evidence region problem.
Experimental results show improved performance on real-world datasets.
Abstract
Evidential deep learning, built upon belief theory and subjective logic, offers a principled and computationally efficient way to turn a deterministic neural network uncertainty-aware. The resultant evidential models can quantify fine-grained uncertainty using the learned evidence. To ensure theoretically sound evidential models, the evidence needs to be non-negative, which requires special activation functions for model training and inference. This constraint often leads to inferior predictive performance compared to standard softmax models, making it challenging to extend them to many large-scale datasets. To unveil the real cause of this undesired behavior, we theoretically investigate evidential models and identify a fundamental limitation that explains the inferior performance: existing evidential activation functions create zero evidence regions, which prevent the model to learn…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsExplainable Artificial Intelligence (XAI) · Adversarial Robustness in Machine Learning · Bayesian Modeling and Causal Inference
MethodsSoftmax
