Generalized Regularized Evidential Deep Learning Models: Theory and Comprehensive Evaluation

TL;DR

This paper introduces a new family of evidential deep learning models with generalized regularizers and activation functions, improving uncertainty quantification and learning dynamics, validated through extensive experiments on multiple benchmarks.

Contribution

It develops a theoretical framework for evidential activations, designs a new family of activation functions and regularizers, and empirically validates their effectiveness across diverse tasks.

Findings

Enhanced uncertainty quantification in neural networks.

Improved learning stability across activation regimes.

Superior performance on multiple benchmark datasets.

Abstract

Evidential deep learning (EDL) models, based on Subjective Logic, introduce a principled and computationally efficient way to make deterministic neural networks uncertainty-aware. The resulting evidential models can quantify fine-grained uncertainty using learned evidence. However, the Subjective-Logic framework constrains evidence to be non-negative, requiring specific activation functions whose geometric properties can induce activation-dependent learning-freeze behavior: a regime where gradients become extremely small for samples mapped into low-evidence regions. We theoretically characterize this behavior and analyze how different evidential activations influence learning dynamics. Building on this analysis, we design a general family of activation functions and corresponding evidential regularizers that provide an alternative pathway for consistent evidence updates across activation regimes. Extensive experiments on four benchmark classification problems (MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet), two few-shot classification problems, and blind face restoration problem empirically validate the developed theory and demonstrate the effectiveness of the proposed generalized regularized evidential models.