Analysing Symbolic Regression Benchmarks under a Meta-Learning Approach

TL;DR

This paper proposes a meta-learning approach to evaluate and improve symbolic regression benchmarks by analyzing dataset meta-features and their correlation with GP performance, revealing current benchmarks' limitations.

Contribution

It introduces a quantitative method using meta-features to assess and guide the selection of diverse datasets for symbolic regression benchmarking.

Findings

Current benchmarks are concentrated in a small region of the meta-feature space.

Number of instances and output skewness are key predictors of GP error.

Meta-features can guide the construction of more effective testbeds.

Abstract

The definition of a concise and effective testbed for Genetic Programming (GP) is a recurrent matter in the research community. This paper takes a new step in this direction, proposing a different approach to measure the quality of the symbolic regression benchmarks quantitatively. The proposed approach is based on meta-learning and uses a set of dataset meta-features---such as the number of examples or output skewness---to describe the datasets. Our idea is to correlate these meta-features with the errors obtained by a GP method. These meta-features define a space of benchmarks that should, ideally, have datasets (points) covering different regions of the space. An initial analysis of 63 datasets showed that current benchmarks are concentrated in a small region of this benchmark space. We also found out that number of instances and output skewness are the most relevant meta-features to GP output error. Both conclusions can help define which datasets should compose an effective testbed for symbolic regression methods.