Improving evidential deep learning via multi-task learning

TL;DR

This paper introduces MT-ENet, a multi-task learning framework that improves the prediction accuracy of evidential regression networks while preserving their uncertainty estimation, by addressing the gradient shrinkage problem with a Lipschitz modified loss.

Contribution

The paper proposes MT-ENet, a novel multi-task learning approach that enhances ENet's accuracy without compromising uncertainty estimation through a dynamic Lipschitz MSE loss.

Findings

MT-ENet improves prediction accuracy on synthetic and real-world datasets.

It maintains the uncertainty estimation capability of the original ENet.

MT-ENet demonstrates superior calibration and out-of-distribution detection on drug-target affinity benchmarks.

Abstract

The Evidential regression network (ENet) estimates a continuous target and its predictive uncertainty without costly Bayesian model averaging. However, it is possible that the target is inaccurately predicted due to the gradient shrinkage problem of the original loss function of the ENet, the negative log marginal likelihood (NLL) loss. In this paper, the objective is to improve the prediction accuracy of the ENet while maintaining its efficient uncertainty estimation by resolving the gradient shrinkage problem. A multi-task learning (MTL) framework, referred to as MT-ENet, is proposed to accomplish this aim. In the MTL, we define the Lipschitz modified mean squared error (MSE) loss function as another loss and add it to the existing NLL loss. The Lipschitz modified MSE loss is designed to mitigate the gradient conflict with the NLL loss by dynamically adjusting its Lipschitz constant. By doing so, the Lipschitz MSE loss does not disturb the uncertainty estimation of the NLL loss. The MT-ENet enhances the predictive accuracy of the ENet without losing uncertainty estimation capability on the synthetic dataset and real-world benchmarks, including drug-target affinity (DTA) regression. Furthermore, the MT-ENet shows remarkable calibration and out-of-distribution detection capability on the DTA benchmarks.