Scale-Parameter Selection in Gaussian Kolmogorov-Arnold Networks

TL;DR

This paper systematically studies the impact of the scale parameter in Gaussian basis functions within Kolmogorov--Arnold Networks, proposing a practical interval for effective scale selection based on first-layer feature analysis.

Contribution

It introduces a stable, effective interval for the scale parameter in Gaussian KANs, emphasizing the importance of the first layer in scale selection and providing a practical design rule.

Findings

Identifies a practical interval for psilon in Gaussian KANs based on first-layer features.

Validates the interval across various approximation and physics-informed problems.

Demonstrates the interval's usefulness for fixed, variable, and constrained scale settings.

Abstract

Kolmogorov--Arnold Networks (KANs) have recently attracted attention as edge-based neural architectures in which learnable univariate functions replace conventional fixed activation functions. A key source of flexibility in KANs is the choice of basis functions used to parameterize the learnable edge functions. In this context, Gaussian basis functions provide a simple and efficient alternative to splines. However, their performance depends strongly on the scale (shape) parameter \(\epsilon\), whose role has not been studied systematically. In this paper, we investigate how \(\epsilon\) affects Gaussian KANs through first-layer feature geometry, conditioning, and approximation behavior. Our central observation is that scale selection is governed primarily by the first layer, since it is the only layer constructed directly on the input domain and any loss of distinguishability introduced there cannot be recovered by later layers. From this viewpoint, we analyze the first-layer feature matrix and identify a practical operating interval, \[ \epsilon \in \left[\frac{1}{G-1},\frac{2}{G-1}\right], \] where \(G\) denotes the number of Gaussian centers. We interpret this interval not as a universal optimality result, but as a stable and effective design rule, and validate it through brute-force sweeps over \(\epsilon\) across function-approximation problems with different collocation densities, grid resolutions, network architectures, and input dimensions, as well as physics-informed problems. We further show that this range is useful for fixed-scale selection, variable-scale constructions, constrained training of \(\epsilon\), and efficient scale search using early training MSE. In this way, the paper positions scale selection as a practical design principle for Gaussian KANs rather than as an ad hoc hyperparameter choice.