Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large $p$

TL;DR

This paper introduces PAN+SR, a scalable variable selection method that improves symbolic regression performance on large input datasets, maintaining accuracy and interpretability in high-dimensional, noisy settings.

Contribution

The paper presents PAN+SR, a novel ab initio nonparametric variable selection approach that enhances existing symbolic regression methods for large-scale, high-dimensional data.

Findings

PAN+SR improves 19 SR methods' performance.

Enables state-of-the-art results on high-dimensional datasets.

Supports scalable, interpretable symbolic modeling.

Abstract

Symbolic regression (SR) is a powerful technique for discovering symbolic expressions that characterize nonlinear relationships in data, gaining increasing attention for its interpretability, compactness, and robustness. However, existing SR methods do not scale to datasets with a large number of input variables (referred to as extreme-scale SR), which is common in modern scientific applications. This ``large $p$'' setting, often accompanied by measurement error, leads to slow performance of SR methods and overly complex expressions that are difficult to interpret. To address this scalability challenge, we propose a method called PAN+SR, which combines a key idea of ab initio nonparametric variable selection with SR to efficiently pre-screen large input spaces and reduce search complexity while maintaining accuracy. The use of nonparametric methods eliminates model misspecification, supporting a strategy called parametric-assisted nonparametric (PAN). We also extend SRBench, an open-source benchmarking platform, by incorporating high-dimensional regression problems with various signal-to-noise ratios. Our results demonstrate that PAN+SR consistently enhances the performance of 19 contemporary SR methods, enabling several to achieve state-of-the-art performance on these challenging datasets.