HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagation

TL;DR

HKAN introduces a hierarchical, non-backpropagation neural network architecture that uses randomized basis functions and linear regression, achieving competitive accuracy with simplified training and enhanced stability.

Contribution

The paper proposes HKAN, a novel hierarchical network that trains without backpropagation by fixing basis parameters and optimizing via least-squares regression, simplifying training and improving stability.

Findings

HKAN achieves comparable or better accuracy than KAN.

HKAN's training is faster and less sensitive to local minima.

HKAN provides insights into variable importance.

Abstract

This paper introduces the Hierarchical Kolmogorov-Arnold Network (HKAN), a novel network architecture that offers a competitive alternative to the recently proposed Kolmogorov-Arnold Network (KAN). Unlike KAN, which relies on backpropagation, HKAN adopts a randomized learning approach, where the parameters of its basis functions are fixed, and linear aggregations are optimized using least-squares regression. HKAN utilizes a hierarchical multi-stacking framework, with each layer refining the predictions from the previous one by solving a series of linear regression problems. This non-iterative training method simplifies computation and eliminates sensitivity to local minima in the loss function. Empirical results show that HKAN delivers comparable, if not superior, accuracy and stability relative to KAN across various regression tasks, while also providing insights into variable importance. The proposed approach seamlessly integrates theoretical insights with practical applications, presenting a robust and efficient alternative for neural network modeling.