Uncertainty Quantification for nonparametric regression using Empirical Bayesian neural networks

TL;DR

This paper introduces a two-step empirical Bayes approach for neural networks that achieves optimal nonparametric regression recovery and reliable uncertainty quantification with faster computation than traditional methods.

Contribution

It presents a novel empirical Bayes method for neural networks that provides optimal recovery and credible sets with frequentist guarantees, requiring only a single fit.

Findings

Achieves near-optimal recovery of the functional parameter

Provides Bayesian credible sets with frequentist coverage

Demonstrates good estimation and uncertainty quantification on synthetic data

Abstract

We propose a new, two-step empirical Bayes-type of approach for neural networks. We show in context of the nonparametric regression model that the procedure (up to a logarithmic factor) provides optimal recovery of the underlying functional parameter of interest and provides Bayesian credible sets with frequentist coverage guarantees. The approach requires fitting the neural network only once, hence it is substantially faster than Bootstrapping type approaches. We demonstrate the applicability of our method over synthetic data, observing good estimation properties and reliable uncertainty quantification.