Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization

TL;DR

This paper enhances the Fast Function Extraction algorithm for symbolic regression by incorporating nonlinear parameter optimization, resulting in more accurate models with minimal runtime increase, validated on the PennML benchmark suite.

Contribution

The paper introduces a novel extension to FFX that optimizes nonlinear parameters, improving accuracy without significantly increasing computational cost.

Findings

Higher accuracy models achieved with the extended FFX

Models of similar length to original FFX

Small increase in runtime for the new method

Abstract

Fast Function Extraction (FFX) is a deterministic algorithm for solving symbolic regression problems. We improve the accuracy of FFX by adding parameters to the arguments of nonlinear functions. Instead of only optimizing linear parameters, we optimize these additional nonlinear parameters with separable nonlinear least squared optimization using a variable projection algorithm. Both FFX and our new algorithm is applied on the PennML benchmark suite. We show that the proposed extensions of FFX leads to higher accuracy while providing models of similar length and with only a small increase in runtime on the given data. Our results are compared to a large set of regression methods that were already published for the given benchmark suite.