Hardware-Oriented Inference Complexity of Kolmogorov-Arnold Networks

TL;DR

This paper introduces platform-independent formulas to evaluate the hardware inference complexity of Kolmogorov-Arnold Networks, facilitating cross-platform comparison and early-stage architectural decisions.

Contribution

It derives generalized, platform-independent metrics for KAN inference complexity, extending analysis to multiple KAN variants and enabling fair comparisons with other neural networks.

Findings

Derived formulas for Real Multiplications, Bit Operations, and Additions/Bit-Shifts.

Extended analysis to B-spline, GRBF, Chebyshev, and Fourier KANs.

Facilitated hardware complexity comparison independent of specific platform metrics.

Abstract

Kolmogorov-Arnold Networks (KANs) have recently emerged as a powerful architecture for various machine learning applications. However, their unique structure raises significant concerns regarding their computational overhead. Existing studies primarily evaluate KAN complexity in terms of Floating-Point Operations (FLOPs) required for GPU-based training and inference. However, in many latency-sensitive and power-constrained deployment scenarios, such as neural network-driven non-linearity mitigation in optical communications or channel state estimation in wireless communications, training is performed offline and dedicated hardware accelerators are preferred over GPUs for inference. Recent hardware implementation studies report KAN complexity using platform-specific resource consumption metrics, such as Look-Up Tables, Flip-Flops, and Block RAMs. However, these metrics require a full hardware design and synthesis stage that limits their utility for early-stage architectural decisions and cross-platform comparisons. To address this, we derive generalized, platform-independent formulae for evaluating the hardware inference complexity of KANs in terms of Real Multiplications (RM), Bit Operations (BOP), and Number of Additions and Bit-Shifts (NABS). We extend our analysis across multiple KAN variants, including B-spline, Gaussian Radial Basis Function (GRBF), Chebyshev, and Fourier KANs. The proposed metrics can be computed directly from the network structure and enable a fair and straightforward inference complexity comparison between KAN and other neural network architectures.