KANs need curvature: penalties for compositional smoothness

TL;DR

This paper introduces a curvature penalty for Kolmogorov-Arnold networks (KANs) to reduce oscillations in activations, enhancing interpretability without sacrificing accuracy.

Contribution

The authors derive a basis-agnostic curvature penalty for KANs, providing theoretical bounds and practical methods to improve interpretability.

Findings

Penalized KANs maintain accuracy while achieving smoother activations.

A curvature upper bound relates model curvature to the penalty.

Richer penalties are motivated to better control curvature.

Abstract

Kolmogorov-Arnold networks (KANs) offer a potent combination of accuracy and interpretability, thanks to their compositions of learnable univariate activation functions. However, the activations of well-fitting KANs tend to exhibit pathologically high-curvature oscillations, making them difficult to interpret, and standard regularization penalties do not prevent this. Here we derive a basis-agnostic curvature penalty and show that penalized models can maintain accuracy while achieving substantially smoother activations. Accounting for how function composition shapes curvature, we prove an upper bound on the full model's curvature relative to the curvature penalty, and use this to motivate richer forms of penalties. Scientific machine learning is increasingly bottlenecked by the trade-off between accuracy and interpretability. Results such as ours that improve interpretability without sacrificing accuracy will further strengthen KANs as a practical tool for both prediction and insight.