Deep Symbolic Regression for Recurrent Sequences

TL;DR

This paper introduces Transformer-based models for symbolic regression of recurrent sequences, successfully predicting underlying functions and relations in integer and float sequences, outperforming traditional methods and providing meaningful approximations.

Contribution

It presents novel Transformer models for symbolic regression of sequences, capable of inferring recurrence relations and functions, including out-of-vocabulary cases, with superior performance.

Findings

Outperforms Mathematica in recurrence prediction on OEIS sequences

Provides meaningful approximations for functions like Bessel and constants like pi^2/6

Demonstrates effectiveness on integer and float sequence modeling

Abstract

Symbolic regression, i.e. predicting a function from the observation of its values, is well-known to be a challenging task. In this paper, we train Transformers to infer the function or recurrence relation underlying sequences of integers or floats, a typical task in human IQ tests which has hardly been tackled in the machine learning literature. We evaluate our integer model on a subset of OEIS sequences, and show that it outperforms built-in Mathematica functions for recurrence prediction. We also demonstrate that our float model is able to yield informative approximations of out-of-vocabulary functions and constants, e.g. $\operatorname{bessel0}(x)\approx \frac{\sin(x)+\cos(x)}{\sqrt{\pi x}}$ and $1.644934\approx \pi^2/6$. An interactive demonstration of our models is provided at https://symbolicregression.metademolab.com.