Active and transfer learning with partially Bayesian neural networks for materials and chemicals

TL;DR

This paper introduces partially Bayesian neural networks (PBNNs) that balance uncertainty quantification and computational efficiency, enhancing active learning for materials and chemicals property prediction.

Contribution

It demonstrates that PBNNs can match fully Bayesian networks in accuracy and uncertainty estimation while reducing computational costs, and leverages pre-trained weights for faster active learning.

Findings

PBNNs achieve comparable accuracy to fully Bayesian networks.

Pre-trained weights improve active learning efficiency.

Validated on molecular and materials science datasets.

Abstract

Active learning, an iterative process of selecting the most informative data points for exploration, is crucial for efficient characterization of materials and chemicals property space. Neural networks excel at predicting these properties but lack the uncertainty quantification needed for active learning-driven exploration. Fully Bayesian neural networks, in which weights are treated as probability distributions inferred via advanced Markov Chain Monte Carlo methods, offer robust uncertainty quantification but at high computational cost. Here, we show that partially Bayesian neural networks (PBNNs), where only selected layers have probabilistic weights while others remain deterministic, can achieve accuracy and uncertainty estimates on active learning tasks comparable to fully Bayesian networks at lower computational cost. Furthermore, by initializing prior distributions with weights pre-trained on theoretical calculations, we demonstrate that PBNNs can effectively leverage computational predictions to accelerate active learning of experimental data. We validate these approaches on both molecular property prediction and materials science tasks, establishing PBNNs as a practical tool for active learning with limited, complex datasets.