Variational Kolmogorov-Arnold Network

TL;DR

InfinityKAN introduces a variational inference method for Kolmogorov-Arnold Networks that automatically learns the number of basis functions, removing the need for manual hyperparameter tuning across diverse datasets.

Contribution

It proposes a novel framework that models the basis count as a latent variable with a prior, enabling automatic capacity adaptation during training.

Findings

Matches or exceeds KAN performance across 18 datasets.

Eliminates manual hyperparameter tuning for basis functions.

Ensures stable training through Lipschitz continuity.

Abstract

Kolmogorov-Arnold Networks (KANs) offer a theoretically grounded alternative to multi-layer perceptrons by representing multivariate functions as compositions of univariate basis functions. However, a critical limitation of KANs is the need to manually specify the number of basis functions per layer -- a hyperparameter that directly controls model capacity and substantially impacts performance, yet whose optimal value varies unpredictably across tasks. We present InfinityKAN, a variational inference framework that eliminates this design choice by learning the number of basis functions during training. Our approach models the basis count as a latent variable with a truncated exponential prior, introducing a differentiable weighting function that enables gradient-based optimization. We establish the Lipschitz continuity of the variational objective, ensuring stable training dynamics. Experiments across 18 datasets spanning synthetic, image, tabular, and graph domains demonstrate that InfinityKAN matches or exceeds the performance of KANs while requiring no manual selection of the number of bases for each layer.