Symbolic Regression via Neural-Guided Genetic Programming Population Seeding

TL;DR

This paper presents a hybrid neural-guided and genetic programming approach for symbolic regression, which improves expression recovery rates and introduces more challenging benchmark problems.

Contribution

A novel hybrid method that uses neural guidance to seed genetic programming populations, enhancing symbolic regression performance.

Findings

Recovered 65% more expressions than previous top models.

Running multiple genetic programming generations without neural guidance performs better.

Introduced 22 new, more difficult benchmark problems.

Abstract

Symbolic regression is the process of identifying mathematical expressions that fit observed output from a black-box process. It is a discrete optimization problem generally believed to be NP-hard. Prior approaches to solving the problem include neural-guided search (e.g. using reinforcement learning) and genetic programming. In this work, we introduce a hybrid neural-guided/genetic programming approach to symbolic regression and other combinatorial optimization problems. We propose a neural-guided component used to seed the starting population of a random restart genetic programming component, gradually learning better starting populations. On a number of common benchmark tasks to recover underlying expressions from a dataset, our method recovers 65% more expressions than a recently published top-performing model using the same experimental setup. We demonstrate that running many genetic programming generations without interdependence on the neural-guided component performs better for symbolic regression than alternative formulations where the two are more strongly coupled. Finally, we introduce a new set of 22 symbolic regression benchmark problems with increased difficulty over existing benchmarks. Source code is provided at www.github.com/brendenpetersen/deep-symbolic-optimization.