Student-t processes as infinite-width limits of posterior Bayesian neural networks

TL;DR

This paper demonstrates that Bayesian neural networks with Gaussian priors and inverse-gamma variance priors converge to Student-t processes in the infinite-width limit, providing a more flexible uncertainty model.

Contribution

It extends the understanding of BNN asymptotics by showing convergence to Student-t processes under specific prior assumptions.

Findings

Posterior BNNs approximate Student-t processes in the infinite-width limit.

The convergence rate is controlled using the Wasserstein metric.

Student-t processes offer greater flexibility than Gaussian processes for modeling uncertainty.

Abstract

The asymptotic properties of Bayesian Neural Networks (BNNs) have been extensively studied, particularly regarding their approximations by Gaussian processes in the infinite-width limit. We extend these results by showing that posterior BNNs can be approximated by Student-t processes, which offer greater flexibility in modeling uncertainty. Specifically, we show that, if the parameters of a BNN follow a Gaussian prior distribution, and the variance of both the last hidden layer and the Gaussian likelihood function follows an Inverse-Gamma prior distribution, then the resulting posterior BNN converges to a Student-t process in the infinite-width limit. Our proof leverages the Wasserstein metric to establish control over the convergence rate of the Student-t process approximation.