Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients

TL;DR

This paper introduces a new deep symbolic regression method that improves robustness and interpretability by using a frequency domain attention decoder, a BIC-based reward, and a ranking policy update, outperforming existing methods.

Contribution

It presents a novel decoder architecture, a BIC-based reward function, and a ranking policy update to enhance deep symbolic regression without relying on pretrained models.

Findings

Outperforms existing DSR methods on benchmarks.

Automatically balances expression complexity and data fit.

Eliminates tail barriers in policy gradient training.

Abstract

We propose a novel deep symbolic regression approach to enhance the robustness and interpretability of data-driven mathematical expression discovery. Our work is aligned with the popular DSR framework which focuses on learning a data-specific expression generator, without relying on pretrained models or additional search or planning procedures. Despite the success of existing DSR methods, they are built on recurrent neural networks, solely guided by data fitness, and potentially meet tail barriers that can zero out the policy gradient, causing inefficient model updates. To overcome these limitations, we design a decoder-only architecture that performs attention in the frequency domain and introduce a dual-indexed position encoding to conduct layer-wise generation. Second, we propose a Bayesian information criterion (BIC)-based reward function that can automatically adjust the trade-off between expression complexity and data fitness, without the need for explicit manual tuning. Third, we develop a ranking-based weighted policy update method that eliminates the tail barriers and enhances training effectiveness. Extensive benchmarks and systematic experiments demonstrate the advantages of our approach. We have released our implementation at https://github.com/ZakBastiani/CADSR.