Softly Symbolifying Kolmogorov-Arnold Networks

TL;DR

This paper introduces S2KAN, a novel neural network model that integrates symbolic primitives into training, enabling interpretable activations while maintaining high accuracy and model efficiency across various tasks.

Contribution

S2KAN is the first model to incorporate differentiable, learnable symbolic primitives directly into neural network training, promoting interpretability and adaptability.

Findings

S2KAN achieves competitive or superior accuracy with smaller models.

It discovers interpretable symbolic forms when possible.

It exhibits emergent self-sparsification without explicit regularization.

Abstract

Kolmogorov-Arnold Networks (KANs) offer a promising path toward interpretable machine learning: their learnable activations can be studied individually, while collectively fitting complex data accurately. In practice, however, trained activations often lack symbolic fidelity, learning pathological decompositions with no meaningful correspondence to interpretable forms. We propose Softly Symbolified Kolmogorov-Arnold Networks (S2KAN), which integrate symbolic primitives directly into training. Each activation draws from a dictionary of symbolic and dense terms, with learnable gates that sparsify the representation. Crucially, this sparsification is differentiable, enabling end-to-end optimization, and is guided by a principled Minimum Description Length objective. When symbolic terms suffice, S2KAN discovers interpretable forms; when they do not, it gracefully degrades to dense splines. We demonstrate competitive or superior accuracy with substantially smaller models across symbolic benchmarks, dynamical systems forecasting, and real-world prediction tasks, and observe evidence of emergent self-sparsification even without regularization pressure.