MetaSymNet: A Tree-like Symbol Network with Adaptive Architecture and Activation Functions

TL;DR

MetaSymNet is an adaptive neural network that dynamically adjusts its structure and employs a novel activation function to produce interpretable mathematical formulas, outperforming existing symbolic regression methods on multiple datasets.

Contribution

The paper introduces MetaSymNet, a neural network with real-time structural adaptation and a unique meta activation function, enabling automatic formula discovery and improved interpretability.

Findings

MetaSymNet outperforms four state-of-the-art symbolic regression algorithms on 10+ datasets.

It achieves better extrapolation and fitting ability than MLP and SVM.

MetaSymNet has simpler network structures compared to traditional MLP with similar performance.

Abstract

Mathematical formulas serve as the means of communication between humans and nature, encapsulating the operational laws governing natural phenomena. The concise formulation of these laws is a crucial objective in scientific research and an important challenge for artificial intelligence (AI). While traditional artificial neural networks (MLP) excel at data fitting, they often yield uninterpretable black box results that hinder our understanding of the relationship between variables x and predicted values y. Moreover, the fixed network architecture in MLP often gives rise to redundancy in both network structure and parameters. To address these issues, we propose MetaSymNet, a novel neural network that dynamically adjusts its structure in real-time, allowing for both expansion and contraction. This adaptive network employs the PANGU meta function as its activation function, which is a unique type capable of evolving into various basic functions during training to compose mathematical formulas tailored to specific needs. We then evolve the neural network into a concise, interpretable mathematical expression. To evaluate MetaSymNet's performance, we compare it with four state-of-the-art symbolic regression algorithms across more than 10 public datasets comprising 222 formulas. Our experimental results demonstrate that our algorithm outperforms others consistently regardless of noise presence or absence. Furthermore, we assess MetaSymNet against MLP and SVM regarding their fitting ability and extrapolation capability, these are two essential aspects of machine learning algorithms. The findings reveal that our algorithm excels in both areas. Finally, we compared MetaSymNet with MLP using iterative pruning in network structure complexity. The results show that MetaSymNet's network structure complexity is obviously less than MLP under the same goodness of fit.