Classified as unknown: A novel Bayesian neural network

TL;DR

This paper introduces a new Bayesian neural network method that estimates output distribution parameters using the probit function, enabling efficient, closed-form Bayesian training and prediction for multi-class neural networks.

Contribution

It develops a novel Bayesian learning algorithm for fully connected neural networks that avoids gradient calculations and Monte Carlo sampling, extending previous perceptron methods to multi-layer, multi-class models.

Findings

Efficient Bayesian training without gradient or sampling.

Closed-form inference for multi-layer neural networks.

Generalization of Bayesian perceptron to multi-class classification.

Abstract

We establish estimations for the parameters of the output distribution for the softmax activation function using the probit function. As an application, we develop a new efficient Bayesian learning algorithm for fully connected neural networks, where training and predictions are performed within the Bayesian inference framework in closed-form. This approach allows sequential learning and requires no computationally expensive gradient calculation and Monte Carlo sampling. Our work generalizes the Bayesian algorithm for a single perceptron for binary classification in \cite{H} to multi-layer perceptrons for multi-class classification.