FC-KAN: Function Combinations in Kolmogorov-Arnold Networks

TL;DR

FC-KAN introduces a novel neural network architecture that combines various mathematical functions to improve performance on image classification tasks, outperforming existing models on MNIST datasets.

Contribution

The paper presents FC-KAN, a new Kolmogorov-Arnold Network that uses function combinations, demonstrating improved accuracy over existing KAN variants on benchmark datasets.

Findings

FC-KAN outperforms other models on MNIST and Fashion-MNIST.

Combining B-splines with Derivative of Gaussians yields the best results.

Function combinations enhance the expressive power of Kolmogorov-Arnold Networks.

Abstract

In this paper, we introduce FC-KAN, a Kolmogorov-Arnold Network (KAN) that leverages combinations of popular mathematical functions such as B-splines, wavelets, and radial basis functions on low-dimensional data through element-wise operations. We explore several methods for combining the outputs of these functions, including sum, element-wise product, the addition of sum and element-wise product, representations of quadratic and cubic functions, concatenation, linear transformation of the concatenated output, and others. In our experiments, we compare FC-KAN with a multi-layer perceptron network (MLP) and other existing KANs, such as BSRBF-KAN, EfficientKAN, FastKAN, and FasterKAN, on the MNIST and Fashion-MNIST datasets. Two variants of FC-KAN, which use a combination of outputs from B-splines and Derivative of Gaussians (DoG) and from B-splines and linear transformations in the form of a quadratic function, outperformed overall other models on the average of 5 independent training runs. We expect that FC-KAN can leverage function combinations to design future KANs. Our repository is publicly available at: https://github.com/hoangthangta/FC_KAN.