Neuro-Symbolic Activation Discovery: Transferring Mathematical Structures from Physics to Ecology for Parameter-Efficient Neural Networks

TL;DR

This paper introduces a neuro-symbolic framework that discovers interpretable activation functions from physics data, enabling parameter-efficient neural networks that generalize across scientific domains, with a focus on physics-ecology transfer.

Contribution

The paper presents a novel method using genetic programming to extract and transfer mathematical activation functions across scientific domains, demonstrating significant parameter efficiency gains.

Findings

Physics-derived activations outperform standard functions in ecology tasks.

Hybrid models achieve similar accuracy with fewer parameters.

Transfer success depends on similarity of underlying mathematical structures.

Abstract

Modern neural networks rely on generic activation functions (ReLU, GELU, SiLU) that ignore the mathematical structure inherent in scientific data. We propose Neuro-Symbolic Activation Discovery, a framework that uses Genetic Programming to extract interpretable mathematical formulas from data and inject them as custom activation functions. Our key contribution is the discovery of a Geometric Transfer phenomenon: activation functions learned from particle physics data successfully generalize to ecological classification, outperforming standard activations (ReLU, GELU, SiLU) in both accuracy and parameter efficiency. On the Forest Cover dataset, our Hybrid Transfer model achieves 82.4% accuracy with only 5,825 parameters, compared to 83.4% accuracy requiring 31,801 parameters for a conventional heavy network -- a 5.5x parameter reduction with only 1% accuracy loss. We introduce a Parameter Efficiency Score ($E_{param} = AUC / \log_{10}(Params)$) and demonstrate that lightweight hybrid architectures consistently achieve 18-21% higher efficiency than over-parameterized baselines. Crucially, we establish boundary conditions: while Physics to Ecology transfer succeeds (both involve continuous Euclidean measurements), Physics to Text transfer fails (discrete word frequencies require different mathematical structures). Our work opens pathways toward domain-specific activation libraries for efficient scientific machine learning.