Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks

TL;DR

This paper introduces a symbolic regression method leveraging vision-enabled large language models and Kolmogorov Arnold Networks, enabling flexible, unbounded function discovery without predefined function sets, and extends to multivariate functions.

Contribution

It presents a novel symbolic regression approach combining LLMs and KANs, removing the need for predefined function sets and enabling multivariate function modeling.

Findings

Effective univariate symbolic regression using LLMs and KANs

Extension to multivariate functions through learned univariate components

Simplification of expressions via language models

Abstract

We present a novel approach to symbolic regression using vision-capable large language models (LLMs) and the ideas behind Google DeepMind's Funsearch. The LLM is given a plot of a univariate function and tasked with proposing an ansatz for that function. The free parameters of the ansatz are fitted using standard numerical optimisers, and a collection of such ans\"atze make up the population of a genetic algorithm. Unlike other symbolic regression techniques, our method does not require the specification of a set of functions to be used in regression, but with appropriate prompt engineering, we can arbitrarily condition the generative step. By using Kolmogorov Arnold Networks (KANs), we demonstrate that ``univariate is all you need'' for symbolic regression, and extend this method to multivariate functions by learning the univariate function on each edge of a trained KAN. The combined expression is then simplified by further processing with a language model.