Layer-wise Lipschitz-Product Control for Deep Kolmogorov--Arnold Network Representations of Compositionally Structured Functions

TL;DR

This paper establishes a layer-wise Lipschitz control framework for deep Kolmogorov-Arnold Networks representing compositional functions, ensuring domain-sensitive bounds and optimal approximation rates.

Contribution

It introduces a novel Lipschitz product control method for deep KANs, addressing previous gaps and providing theoretical bounds and experimental validation.

Findings

Lipschitz product P(KAN) <= 1 for standard operations.

Bounded approximation error proportional to N and epsilon_Op.

Experiments confirm P(KAN)=1.0 for structured functions.

Abstract

We prove that any continuous function f from [0,1]^n to R representable by a finite computation tree with N internal nodes and compositional sparsity s = O(1) admits a deep Kolmogorov-Arnold Network (KAN) representation. Each internal node is realised by a primitive KAN block with controlled block depth and Lipschitz product. The layer-wise Lipschitz product satisfies the primary domain-sensitive bound independent of the input dimension n. It simplifies to P(KAN_f) <= max(C*,1)^L_f with L_f <= c_max * N. For the standard operations {+,-,x,sin,cos} with x nodes on [0,1]-bounded inputs we obtain P(KAN) <= 1. Layer widths satisfy n_l <= n + 2 w_max * N. The uniform approximation error is bounded by N * max(C*,1)^d(f) * epsilon_Op (simplifies when C* <=1). For f in C^m we obtain optimal B-spline rates. Range bounds are also derived (B_f <= N+1 for additive trees). This addresses the gap on Lipschitz control in deep KAN stacks noted by Liu et al. (2024). Experiments confirm P(KAN)=1.0 for several compositionally structured functions.