Scalable and Interpretable Scientific Discovery via Sparse Variational Gaussian Process Kolmogorov-Arnold Networks (SVGP KAN)

TL;DR

This paper introduces SVGP-KAN, a scalable probabilistic neural network architecture that combines sparse variational Gaussian processes with Kolmogorov-Arnold Networks, improving interpretability and enabling analysis of large scientific datasets.

Contribution

The paper presents SVGP-KAN, a novel scalable probabilistic KAN architecture using sparse variational inference, allowing application to larger datasets and structural input analysis.

Findings

Reduced computational complexity from O(N^3) to O(NM^2)

Enabled probabilistic KANs on larger datasets

Integrated permutation-based importance analysis for interpretability

Abstract

Kolmogorov-Arnold Networks (KANs) offer a promising alternative to Multi-Layer Perceptron (MLP) by placing learnable univariate functions on network edges, enhancing interpretability. However, standard KANs lack probabilistic outputs, limiting their utility in applications requiring uncertainty quantification. While recent Gaussian Process (GP) extensions to KANs address this, they utilize exact inference methods that scale cubically with data size N, restricting their application to smaller datasets. We introduce the Sparse Variational GP-KAN (SVGP-KAN), an architecture that integrates sparse variational inference with the KAN topology. By employing $M$ inducing points and analytic moment matching, our method reduces computational complexity from $O(N^3)$ to $O(NM^2)$ or linear in sample size, enabling the application of probabilistic KANs to larger scientific datasets. Furthermore, we demonstrate that integrating a permutation-based importance analysis enables the network to function as a framework for structural identification, identifying relevant inputs and classifying functional relationships.