Variational Bayes Neural Network: Posterior Consistency, Classification Accuracy and Computational Challenges

TL;DR

This paper introduces a variational Bayesian neural network methodology with theoretical guarantees for posterior consistency, addressing computational challenges and improving prediction accuracy in complex biomedical applications.

Contribution

It develops a new variational Bayesian neural network approach with theoretical analysis, including posterior consistency and guidelines for prior and variational family selection.

Findings

Proposes a scalable variational Bayesian neural network method.

Provides theoretical guarantees for posterior consistency.

Quantifies the loss of using variational posterior compared to true posterior.

Abstract

Bayesian neural network models (BNN) have re-surged in recent years due to the advancement of scalable computations and its utility in solving complex prediction problems in a wide variety of applications. Despite the popularity and usefulness of BNN, the conventional Markov Chain Monte Carlo based implementation suffers from high computational cost, limiting the use of this powerful technique in large scale studies. The variational Bayes inference has become a viable alternative to circumvent some of the computational issues. Although the approach is popular in machine learning, its application in statistics is somewhat limited. This paper develops a variational Bayesian neural network estimation methodology and related statistical theory. The numerical algorithms and their implementational are discussed in detail. The theory for posterior consistency, a desirable property in nonparametric Bayesian statistics, is also developed. This theory provides an assessment of prediction accuracy and guidelines for characterizing the prior distributions and variational family. The loss of using a variational posterior over the true posterior has also been quantified. The development is motivated by an important biomedical engineering application, namely building predictive tools for the transition from mild cognitive impairment to Alzheimer's disease. The predictors are multi-modal and may involve complex interactive relations.