Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search

TL;DR

This paper introduces a novel Monte-Carlo Tree Search method combined with neural mutation models for symbolic regression, achieving state-of-the-art results by integrating neural network speed with search capabilities.

Contribution

It proposes a new hybrid approach that combines neural models with Monte-Carlo Tree Search to improve symbolic regression performance, especially on out-of-distribution data.

Findings

Achieves state-of-the-art results on SRBench benchmark.

Combines neural mutation models with Monte-Carlo Tree Search.

Outperforms traditional genetic programming methods.

Abstract

Symbolic regression (SR) is the problem of learning a symbolic expression from numerical data. Recently, deep neural models trained on procedurally-generated synthetic datasets showed competitive performance compared to more classical Genetic Programming (GP) algorithms. Unlike their GP counterparts, these neural approaches are trained to generate expressions from datasets given as context. This allows them to produce accurate expressions in a single forward pass at test time. However, they usually do not benefit from search abilities, which result in low performance compared to GP on out-of-distribution datasets. In this paper, we propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure using a context-aware neural mutation model, which is initially pre-trained to learn promising mutations, and further refined from successful experiences in an online fashion. The approach demonstrates state-of-the-art performance on the well-known \texttt{SRBench} benchmark.