Learning Linear Feature Space Transformations in Symbolic Regression

TL;DR

This paper introduces a novel linear feature space transformation node for symbolic regression, exploring different synchronization modes and weight evolution methods, leading to improved performance in benchmark tests.

Contribution

It proposes a new linear combination leaf node for symbolic regression with multiple synchronization modes and weight evolution methods, enhancing MGGP performance.

Findings

Two configurations improved algorithm performance

Different synchronization modes affect results

Gradient-based weight evolution is effective

Abstract

We propose a new type of leaf node for use in Symbolic Regression (SR) that performs linear combinations of feature variables (LCF). These nodes can be handled in three different modes -- an unsynchronized mode, where all LCFs are free to change on their own, a synchronized mode, where LCFs are sorted into groups in which they are forced to be identical throughout the whole individual, and a globally synchronized mode, which is similar to the previous mode but the grouping is done across the whole population. We also present two methods of evolving the weights of the LCFs -- a purely stochastic way via mutation and a gradient-based way based on the backpropagation algorithm known from neural networks -- and also a combination of both. We experimentally evaluate all configurations of LCFs in Multi-Gene Genetic Programming (MGGP), which was chosen as baseline, on a number of benchmarks. According to the results, we identified two configurations which increase the performance of the algorithm.