Deep Variational Inference Symbolic Regression

TL;DR

DVISR extends deep symbolic regression with variational Bayesian methods, enabling uncertainty quantification over symbolic expressions and constants, advancing scalable Bayesian symbolic regression.

Contribution

Introduces DVISR, a variational Bayesian extension of deep symbolic regression that infers posterior distributions over expressions and constants.

Findings

DVISR can recover true posterior in simple settings.

Performance varies with size of expression space.

Enables uncertainty quantification in symbolic regression.

Abstract

Symbolic regression discovers explicit, interpretable equations without assuming a functional form in advance. A Bayesian approach strengthens this through probability distributions over candidate expressions, thus quantifying uncertainty in the presence of noisy and limited data. Deep Symbolic Regression (DSR) uses a neural network to generate symbolic expressions, but it is designed to identify a single best-fitting expression rather than infer a posterior distribution over models. We introduce Deep Variational Inference Symbolic Regression (DVISR), a variational Bayesian extension of DSR. DVISR replaces the original reward with the integrand of the evidence lower bound. It also extends the network architecture to output distributions over constants within expressions, enabling posterior inference over both expression trees and their associated constants. We show that DVISR can recover the true posterior in simple settings, both with and without constant tokens, and we examine how its performance changes as the size of the expression space increases. These results position DVISR as a step toward scalable Bayesian symbolic regression with uncertainty over full symbolic models.