cp3-bench: A tool for benchmarking symbolic regression algorithms tested with cosmology

TL;DR

cp3-bench is a user-friendly, open-source benchmarking tool for symbolic regression algorithms, tested on astrophysical datasets, revealing insights into factors affecting algorithm performance and guiding better algorithm selection.

Contribution

Introduces cp3-bench, a comprehensive, easy-to-use benchmarking platform for symbolic regression algorithms, including a comparative study on astrophysical datasets to inform algorithm development and selection.

Findings

Most algorithms perform poorly on the benchmark.

Feature space dimensionality and dataset precision are crucial for success.

Inter-dependence of features does not hinder algorithm performance.

Abstract

We introduce cp3-bench, a tool for comparing symbolic regression algorithms which we make publicly available at https://github.com/CP3-Origins/cp3-bench. Currently, cp3-bench includes 12 symbolic regression algorithms which can be automatically installed as part of cp3-bench. The philosophy behind cp3-bench is that it should be as user-friendly as possible, available in a ready-to-use format, and allow for easy additions of new algorithms and datasets. Our hope is that users of symbolic regression algorithms can use cp3-bench to easily install and compare symbolic regression algorithms to better decide which algorithms to use for their specific tasks at hand. To introduce and motivate the use of cp3-bench we present a benchmark of 12 symbolic regression algorithms applied to 28 datasets representing six different astrophysical setups. Overall, we find that most of the benched algorithms do poorly in the benchmark and suggest possible ways to proceed with developing algorithms that will be better at identifying ground truth expressions for cosmological datasets. Our demonstration benchmark studies the significance of dimensionality of the feature space and precision of datasets. We find both to be highly important for symbolic regression tasks to be successful. On the other hand, we find no indication that inter-dependence of features in datasets is important, i.e. it is not in general a hindrance for symbolic regression algorithms if datasets e.g. contain both $z$ and $H(z)$ as features. Lastly, we find no indication that performance of algorithms on standardized datasets are good indicators of performance on particular astrophysical datasets. This suggests that it is not necessarily prudent to choose symbolic regression algorithms based on their performance on standardized data. A more robust approach is to consider a variety of algorithms, chosen based on the particular task at hand.