Towards Expressive Priors for Bayesian Neural Networks: Poisson Process Radial Basis Function Networks

TL;DR

This paper introduces Poisson Process Radial Basis Function Networks, a new Bayesian prior that allows flexible, decoupled specification of amplitude and lengthscale, improving expressiveness and consistency in neural network modeling.

Contribution

The paper proposes a novel prior for Bayesian neural networks that encodes amplitude stationarity and input-dependent lengthscale, with proven consistency and practical demonstration.

Findings

Allows decoupled specification of amplitude and lengthscale

Ensures consistent regression function estimation

Demonstrates effectiveness on synthetic and real data

Abstract

While Bayesian neural networks have many appealing characteristics, current priors do not easily allow users to specify basic properties such as expected lengthscale or amplitude variance. In this work, we introduce Poisson Process Radial Basis Function Networks, a novel prior that is able to encode amplitude stationarity and input-dependent lengthscale. We prove that our novel formulation allows for a decoupled specification of these properties, and that the estimated regression function is consistent as the number of observations tends to infinity. We demonstrate its behavior on synthetic and real examples.