UniSymNet: A Unified Symbolic Network Guided by Transformer

TL;DR

UniSymNet introduces a unified symbolic network guided by a Transformer, effectively reducing complexity and overfitting in symbolic regression, and demonstrates high accuracy and symbolic solution rates on benchmark datasets.

Contribution

It unifies binary operators into nested unary operators and employs a Transformer-guided approach for structural selection, advancing symbolic regression methods.

Findings

High fitting accuracy on benchmarks

Excellent symbolic solution rate

Relatively low expression complexity

Abstract

Symbolic Regression (SR) is a powerful technique for automatically discovering mathematical expressions from input data. Mainstream SR algorithms search for the optimal symbolic tree in a vast function space, but the increasing complexity of the tree structure limits their performance. Inspired by neural networks, symbolic networks have emerged as a promising new paradigm. However, most existing symbolic networks still face certain challenges: binary nonlinear operators $\{\times, \div\}$ cannot be naturally extended to multivariate operators, and training with fixed architecture often leads to higher complexity and overfitting. In this work, we propose a Unified Symbolic Network that unifies nonlinear binary operators into nested unary operators and define the conditions under which UniSymNet can reduce complexity. Moreover, we pre-train a Transformer model with a novel label encoding method to guide structural selection, and adopt objective-specific optimization strategies to learn the parameters of the symbolic network. UniSymNet shows high fitting accuracy, excellent symbolic solution rate, and relatively low expression complexity, achieving competitive performance on low-dimensional Standard Benchmarks and high-dimensional SRBench.