TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functions

TL;DR

TruKAN introduces a novel architecture using truncated power functions within Kolmogorov-Arnold Networks, enhancing interpretability, accuracy, and efficiency in computer vision tasks by replacing traditional basis functions.

Contribution

The paper presents TruKAN, a new KAN-based model employing truncated power functions, improving interpretability, training efficiency, and accuracy over existing KAN variants.

Findings

TruKAN outperforms other KAN models in accuracy and efficiency.

Layer normalization impacts model performance.

Shared versus individual knots affect interpretability and accuracy.

Abstract

To address the trade-off between computational efficiency and adherence to Kolmogorov-Arnold Network (KAN) principles, we propose TruKAN, a new architecture based on the KAN structure and learnable activation functions. TruKAN replaces the B-spline basis in KAN with a family of truncated power functions derived from k-order spline theory. This change maintains the KAN's expressiveness while enhancing accuracy and training time. Each TruKAN layer combines a truncated power term with a polynomial term and employs either shared or individual knots. TruKAN exhibits greater interpretability than other KAN variants due to its simplified basis functions and knot configurations. By prioritizing interpretable basis functions, TruKAN aims to balance approximation efficacy with transparency. We develop the TruKAN model and integrate it into an advanced EfficientNet-V2-based framework, which is then evaluated on computer vision benchmark datasets. To ensure a fair comparison, we develop various models: MLP-, KAN-, SineKAN and TruKAN-based EfficientNet frameworks and assess their training time and accuracy across small and deep architectures. The training phase uses hybrid optimization to improve convergence stability. Additionally, we investigate layer normalization techniques for all the models and assess the impact of shared versus individual knots in TruKAN. Overall, TruKAN outperforms other KAN models in terms of accuracy, computational efficiency and memory usage on the complex vision task, demonstrating advantages beyond the limited settings explored in prior KAN studies.