Automated Learning of Interpretable Models with Quantified Uncertainty

TL;DR

This paper introduces a Bayesian genetic programming framework for symbolic regression that quantifies uncertainty, enhances interpretability, and improves robustness to noise in machine learning models.

Contribution

It presents a novel Bayesian approach to GPSR that uses model evidence for selection, enabling automatic uncertainty quantification and improved interpretability.

Findings

Quantifies model parameter uncertainty for probabilistic predictions.

Increases interpretability and robustness to noise.

Reduces overfitting compared to conventional GPSR.

Abstract

Interpretability and uncertainty quantification in machine learning can provide justification for decisions, promote scientific discovery and lead to a better understanding of model behavior. Symbolic regression provides inherently interpretable machine learning, but relatively little work has focused on the use of symbolic regression on noisy data and the accompanying necessity to quantify uncertainty. A new Bayesian framework for genetic-programming-based symbolic regression (GPSR) is introduced that uses model evidence (i.e., marginal likelihood) to formulate replacement probability during the selection phase of evolution. Model parameter uncertainty is automatically quantified, enabling probabilistic predictions with each equation produced by the GPSR algorithm. Model evidence is also quantified in this process, and its use is shown to increase interpretability, improve robustness to noise, and reduce overfitting when compared to a conventional GPSR implementation on both numerical and physical experiments.