GESR: A Genetic Programming-Based Symbolic Regression Method with Gene Editing

TL;DR

The paper introduces GESR, a symbolic regression method using gene editing guided by BERT models, which enhances efficiency and performance over traditional genetic programming approaches.

Contribution

It presents a novel gene editing approach for symbolic regression guided by BERT models, improving efficiency and effectiveness compared to standard genetic programming methods.

Findings

GESR significantly outperforms traditional GP in computational efficiency.

GESR achieves strong results across multiple symbolic regression tasks.

The method demonstrates the potential of language models in guiding genetic algorithms.

Abstract

Mathematical formulas serve as a language through which humans communicate with nature. Discovering mathematical laws from scientific data to describe natural phenomena has been a long-standing pursuit of humanity for centuries. In the field of artificial intelligence, this challenge is known as the symbolic regression problem. Among existing symbolic regression approaches, Genetic Programming (GP) based on evolutionary algorithms remains one of the most classical and widely adopted methods. GP simulates the evolutionary process across generations through genetic mutation and crossover. However, mutations and crossovers in GP are entirely random. While this randomness effectively mimics natural evolution, it inevitably produces both beneficial and detrimental variations. If there existed a metaphorical `God` capable of foreseeing which genetic mutations or crossovers would yield superior outcomes and performing targeted gene editing accordingly, the efficiency of evolution could be substantially improved. Motivated by this idea, we propose in this paper a symbolic regression approach based on gene editing, termed GESR. In GESR, we trained two "hands of God" (two BERT models). Among them, the first leverages the BERT's masked language modeling capability to guide the mutation of genes (expression symbols). The other BERT model guides the crossover of individual genes by predicting the crossover point. Experimental results demonstrate that GESR significantly improves computational efficiency compared with traditional GP algorithms and achieves strong overall performance across multiple symbolic regression tasks.