KAN/H: Kolmogorov-Arnold Network using Haar-like bases

TL;DR

This paper introduces KAN/H, a novel hierarchical basis system for function approximation that improves efficiency and flexibility, especially in high-dimensional settings, by using Haar-like bases with specific derivative properties.

Contribution

The paper presents KAN/H, a new variant of Kolmogorov-Arnold Network utilizing Haar-like bases with nonzero derivatives, along with a learning-rate schedule and input handling techniques.

Findings

Effective in high-dimensional function approximation

Requires minimal hyperparameter tuning

Successfully applied to MNIST dataset

Abstract

Function approximation using Haar basis systems offers an efficient implementation when compressed via Patricia trees while retaining the flexibility of wavelets for both global and local fitting. However, like B-spline-based approximations, achieving high accuracy in high dimensions remains challenging. This paper proposes KAN/H, a variant of the Kolmogorov-Arnold Network (KAN) that uses a Haar-like hierarchical basis system with nonzero first-order derivatives, instead of B-splines. We also propose a learning-rate scheduling method and a method for handling unbounded real-valued inputs, leveraging properties of linear approximation with Haar-like hierarchical bases. By applying the resulting algorithm to function-approximation problems and MNIST, we confirm that our approach requires minimal problem-specific hyperparameter tuning.