Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning

TL;DR

This paper presents a method to decompose uncertainty in Bayesian neural networks into epistemic and aleatoric parts, enabling improved active learning and risk-sensitive decision-making in reinforcement learning.

Contribution

It introduces a novel uncertainty decomposition technique for Bayesian neural networks and applies it to active learning and risk-sensitive policy optimization.

Findings

Effective identification of informative points for active learning.

Successful application of uncertainty decomposition in risk-sensitive reinforcement learning.

Enhanced decision-making by balancing cost, bias, and noise.

Abstract

Bayesian neural networks with latent variables are scalable and flexible probabilistic models: They account for uncertainty in the estimation of the network weights and, by making use of latent variables, can capture complex noise patterns in the data. We show how to extract and decompose uncertainty into epistemic and aleatoric components for decision-making purposes. This allows us to successfully identify informative points for active learning of functions with heteroscedastic and bimodal noise. Using the decomposition we further define a novel risk-sensitive criterion for reinforcement learning to identify policies that balance expected cost, model-bias and noise aversion.