Programmatic Context Augmentation for LLM-based Symbolic Regression

TL;DR

This paper introduces a new LLM-based evolutionary framework for symbolic regression that enhances data interaction through programmatic context augmentation, leading to improved efficiency and accuracy.

Contribution

It presents a novel approach that integrates code-based dataset interactions into LLM-driven symbolic regression, overcoming limitations of scalar evaluation metrics.

Findings

Outperforms existing methods on LLM-SRBench benchmark.

Achieves higher accuracy and efficiency in symbolic regression tasks.

Utilizes programmatic data analysis to extract richer signals.

Abstract

Symbolic regression (SR), the task of discovering mathematical expressions that best describe a given dataset, remains a fundamental challenge in scientific discovery. Traditional approaches, primarily based on genetic algorithms and related evolutionary methods, have proven useful but suffer from scalability and expressivity limitations. Recently, large language model (LLM)-based evolutionary search methods have been introduced into SR and show promise. However, existing LLM-based approaches typically rely on scalar evaluation metrics, such as mean squared error, as the sole source of feedback during the search process, thereby overlooking the rich information embedded in the dataset. To address this limitation, we propose a novel LLM-based evolutionary search framework that incorporates programmatic context augmentation. By enabling code-based interactions with the dataset, our method can actively perform data analysis and extract informative signals, beyond aggregated evaluation scores. We evaluate our framework on advanced benchmarks, such as LLM-SRBench, and demonstrate superior efficiency and accuracy compared to strong baselines.