Augmenting Neural Networks with Priors on Function Values

TL;DR

This paper introduces a probabilistic method to incorporate prior knowledge of function values into Bayesian neural networks, enhancing their accuracy especially in regions with high uncertainty, which is valuable in scientific applications with limited data.

Contribution

We develop a novel approach to augment Bayesian neural networks with direct priors on function values, improving their performance in data-scarce scientific domains.

Findings

The method effectively combines prior knowledge with neural network predictions.

Predictions rely more on priors in high-uncertainty regions.

The approach improves function estimation in label-limited settings.

Abstract

The need for function estimation in label-limited settings is common in the natural sciences. At the same time, prior knowledge of function values is often available in these domains. For example, data-free biophysics-based models can be informative on protein properties, while quantum-based computations can be informative on small molecule properties. How can we coherently leverage such prior knowledge to help improve a neural network model that is quite accurate in some regions of input space -- typically near the training data -- but wildly wrong in other regions? Bayesian neural networks (BNN) enable the user to specify prior information only on the neural network weights, not directly on the function values. Moreover, there is in general no clear mapping between these. Herein, we tackle this problem by developing an approach to augment BNNs with prior information on the function values themselves. Our probabilistic approach yields predictions that rely more heavily on the prior information when the epistemic uncertainty is large, and more heavily on the neural network when the epistemic uncertainty is small.