A Neural Bayesian Estimator for Conditional Probability Densities

TL;DR

This paper introduces a neural Bayesian method for non-parametric estimation of conditional probability densities, enabling flexible, event-by-event predictions with automatic handling of non-Gaussian tails and no distributional assumptions.

Contribution

It presents a novel neural network-based Bayesian algorithm for estimating conditional probability densities, applicable to complex problems with weakly correlated variables.

Findings

Effective in modeling non-Gaussian tails

Automatically derives moments and expectation values

Demonstrated on high-energy physics and econometrics data

Abstract

This article describes a robust algorithm to estimate a conditional probability density f(t|x) as a non-parametric smooth regression function. It is based on a neural network and the Bayesian interpretation of the network output as a posteriori probabability. The network is trained using example events from history or simulation, which define the underlying probability density f(t,x). Once trained, the network is applied on new, unknown examples x, for which it can predict the probability distribution of the target variable t. Event-by-event knowledge of the smooth function f(t|x) can be very useful, e.g. in maximum likelihood fits or for forecasting tasks. No assumptions are necessary about the distribution, and non-Gaussian tails are accounted for automatically. Important quantities like median, mean value, left and right standard deviations, moments and expectation values of any function of t are readily derived from it. The algorithm can be considered as an event-by-event unfolding and leads to statistically optimal reconstruction. The largest benefit of the method lies in complicated problems, when the measurements x are only relatively weakly correlated to the output t. As to assure optimal generalisation features and to avoid overfitting, the networks are regularised by extended versions of weight decay. The regularisation parameters are determined during the online-learning of the network by relations obtained from Bayesian statistics. Some toy Monte Carlo tests and first real application examples from high-energy physics and econometry are discussed.