Shape Constraints in Symbolic Regression using Penalized Least Squares

TL;DR

This paper introduces a method for incorporating shape constraints into symbolic regression by minimizing violations during parameter identification, improving performance especially with limited data.

Contribution

It proposes a novel gradient-based approach to enforce shape constraints during parameter optimization in symbolic regression, unlike previous methods.

Findings

Shape constraints improve symbolic regression with scarce data.

Minimizing violations during parameter identification outperforms constraint-only methods.

The approach is statistically beneficial in several test cases.

Abstract

We study the addition of shape constraints (SC) and their consideration during the parameter identification step of symbolic regression (SR). SC serve as a means to introduce prior knowledge about the shape of the otherwise unknown model function into SR. Unlike previous works that have explored SC in SR, we propose minimizing SC violations during parameter identification using gradient-based numerical optimization. We test three algorithm variants to evaluate their performance in identifying three symbolic expressions from synthetically generated data sets. This paper examines two benchmark scenarios: one with varying noise levels and another with reduced amounts of training data. The results indicate that incorporating SC into the expression search is particularly beneficial when data is scarce. Compared to using SC only in the selection process, our approach of minimizing violations during parameter identification shows a statistically significant benefit in some of our test cases, without being significantly worse in any instance.