Bayesian inference with sources of uncertainty: from confidence modelling to sparse estimation

TL;DR

This paper presents a flexible Bayesian inference framework that explicitly encodes confidence in uncertainty sources, enabling improved model design, regularisation, and sparsity in various models including neural networks.

Contribution

It introduces a novel framework for confidence encoding in Bayesian models, facilitating sparsity and regularisation in linear, logistic regression, and neural networks.

Findings

Framework allows explicit confidence encoding in models

Enables sparsity in linear, logistic regression, and neural networks

Provides new regularisation control mechanisms

Abstract

We introduce a general framework that extends Bayesian inference by allowing the researcher to explicitly encode confidence in each source of uncertainty within the model. This mechanism provides a new handle for model design and regularisation control. Building on this framework, we develop a general approach for inducing sparsity in statistical models and illustrate its use in linear and logistic regression, as well as in Bayesian neural networks.