KAConvNet: Kolmogorov-Arnold Convolutional Networks for Vision Recognition

TL;DR

KAConvNet introduces a novel convolutional layer based on the Kolmogorov-Arnold theorem, enhancing interpretability and efficiency in vision recognition tasks, and outperforms existing KAN-based models.

Contribution

The paper proposes a new Kolmogorov-Arnold convolutional layer integrated with CNNs, improving interpretability and performance over previous KAN approaches.

Findings

KAConvNet outperforms existing KAN-based methods.

The model achieves competitive results with mainstream ViTs and CNNs.

The code is publicly available at https://github.com/UnicomAI/KAConvNet.

Abstract

The Convolutional Neural Networks (CNNs) have been the dominant and effective approach for general computer vision tasks. Recently, Kolmogorov-Arnold neural networks (KANs), based on the Kolmogorov-Arnold representation theorem, have shown potential to replace Multi-Layer Perceptrons (MLPs) in deep learning. KANs, which use learnable nonlinear activations on edges and simple summation on nodes, offer fewer parameters and greater explainability compared to MLPs. However, there has been limited exploration of integrating the Kolmogorov-Arnold representation theorem with convolutional methods for computer vision tasks. Existing attempts have merely replaced learnable activation functions with weights, undermining KANs' theoretical foundation and limiting their potential effectiveness. Additionally, the B-spline curves used in KANs suffer from computational inefficiency and a tendency to overfit. In this paper, we propose a novel Kolmogorov-Arnold Convolutional Layer that deeply integrates the Kolmogorov-Arnold representation theorem with convolution. This layer provides stronger method interpretability because it is based on established mathematical theorems and its design has theoretical alignment. Building on the Kolmogorov-Arnold Convolutional Layer, we design an efficient network architecture called KAConvNet, which outperforms existing methods combining KAN and convolution, and achieves competitive performance compared to mainstream ViTs and CNNs. We believe that our work offers valuable insight into the field of artificial intelligence and will inspire the development of more innovative CNNs in the 2020s. The code is publicly available at https://github.com/UnicomAI/KAConvNet.