Sparse Interpretable Deep Learning with LIES Networks for Symbolic Regression

TL;DR

This paper introduces LIES networks, a novel neural architecture with interpretable activations designed for scalable, accurate, and sparse symbolic regression, outperforming existing methods.

Contribution

The paper presents a fixed neural network architecture with primitive activations and a training framework for extracting compact symbolic expressions, improving scalability and interpretability in symbolic regression.

Findings

LIES networks produce sparse, accurate symbolic formulas.

LIES outperforms baseline methods on SR benchmarks.

Ablation studies confirm the effectiveness of design components.

Abstract

Symbolic regression (SR) aims to discover closed-form mathematical expressions that accurately describe data, offering interpretability and analytical insight beyond standard black-box models. Existing SR methods often rely on population-based search or autoregressive modeling, which struggle with scalability and symbolic consistency. We introduce LIES (Logarithm, Identity, Exponential, Sine), a fixed neural network architecture with interpretable primitive activations that are optimized to model symbolic expressions. We develop a framework to extract compact formulae from LIES networks by training with an appropriate oversampling strategy and a tailored loss function to promote sparsity and to prevent gradient instability. After training, it applies additional pruning strategies to further simplify the learned expressions into compact formulae. Our experiments on SR benchmarks show that the LIES framework consistently produces sparse and accurate symbolic formulae outperforming all baselines. We also demonstrate the importance of each design component through ablation studies.