Gaussian Process Kolmogorov-Arnold Networks

TL;DR

This paper introduces GP-KAN, a probabilistic neural network combining Gaussian Processes with Kolmogorov Arnold Networks, offering robust non-linear modeling, uncertainty estimation, and efficient training, demonstrated on MNIST with high accuracy and fewer parameters.

Contribution

The paper presents a novel probabilistic extension of KANs using Gaussian Processes, enabling analytical output distributions, uncertainty quantification, and efficient training without variational methods.

Findings

Achieved 98.5% accuracy on MNIST with only 80k parameters.

Provided a fully analytical approach to GP-based neural networks.

Demonstrated competitive performance with significantly fewer parameters.

Abstract

In this paper, we introduce a probabilistic extension to Kolmogorov Arnold Networks (KANs) by incorporating Gaussian Process (GP) as non-linear neurons, which we refer to as GP-KAN. A fully analytical approach to handling the output distribution of one GP as an input to another GP is achieved by considering the function inner product of a GP function sample with the input distribution. These GP neurons exhibit robust non-linear modelling capabilities while using few parameters and can be easily and fully integrated in a feed-forward network structure. They provide inherent uncertainty estimates to the model prediction and can be trained directly on the log-likelihood objective function, without needing variational lower bounds or approximations. In the context of MNIST classification, a model based on GP-KAN of 80 thousand parameters achieved 98.5% prediction accuracy, compared to current state-of-the-art models with 1.5 million parameters.