Efficient Generator of Mathematical Expressions for Symbolic Regression

TL;DR

This paper introduces HVAE, a variational autoencoder that efficiently encodes mathematical expressions into a smooth latent space, enabling improved symbolic regression through optimization and evolutionary algorithms.

Contribution

The novel HVAE model effectively encodes hierarchical mathematical expressions, facilitating better exploration of the expression space for symbolic regression tasks.

Findings

HVAE can be trained with small datasets of expressions.

Latent space exploration outperforms probabilistic grammar methods.

EDHiE system surpasses state-of-the-art in symbolic regression benchmarks.

Abstract

We propose an approach to symbolic regression based on a novel variational autoencoder for generating hierarchical structures, HVAE. It combines simple atomic units with shared weights to recursively encode and decode the individual nodes in the hierarchy. Encoding is performed bottom-up and decoding top-down. We empirically show that HVAE can be trained efficiently with small corpora of mathematical expressions and can accurately encode expressions into a smooth low-dimensional latent space. The latter can be efficiently explored with various optimization methods to address the task of symbolic regression. Indeed, random search through the latent space of HVAE performs better than random search through expressions generated by manually crafted probabilistic grammars for mathematical expressions. Finally, EDHiE system for symbolic regression, which applies an evolutionary algorithm to the latent space of HVAE, reconstructs equations from a standard symbolic regression benchmark better than a state-of-the-art system based on a similar combination of deep learning and evolutionary algorithms.\v{z}