Necessary and sufficient conditions for universality of Kolmogorov-Arnold networks

TL;DR

This paper establishes the necessary and sufficient conditions for the universality of Kolmogorov-Arnold Networks (KANs), showing that a single non-affine edge function suffices for universal approximation.

Contribution

It provides a complete characterization of when KANs are universal, including the minimal set of edge functions needed and the universality of spline-based KANs.

Findings

A single non-affine edge function ensures universality in deep KANs.

Two hidden layers require the edge function to be nonpolynomial for universality.

A finite set of affine functions combined with a non-affine function suffices for universality.

Abstract

We analyze the universal approximation property of Kolmogorov-Arnold Networks (KANs) in terms of their edge functions. If these functions are all affine, then universality clearly fails. How many non-affine functions are needed, in addition to affine ones, to ensure universality? We show that a single one suffices. More precisely, we prove that deep KANs in which all edge functions are either affine or equal to a fixed continuous function $\sigma$ are dense in $C(K)$ for every compact set $K\subset\mathbb{R}^n$ if and only if $\sigma$ is non-affine. In contrast, for KANs with exactly two hidden layers, universality holds if and only if $\sigma$ is nonpolynomial. We further show that the full class of affine functions is not required; it can be replaced by a finite set without affecting universality. In particular, in the nonpolynomial case, a fixed family of five affine functions suffices when the depth is arbitrary. More generally, for every continuous non-affine function $\sigma$, there exists a finite affine family $A_\sigma$ such that deep KANs with edge functions in $A_\sigma\cup\{\sigma\}$ remain universal. We also prove that KANs with the spline-based edge parameterization introduced by Liu et al.~\cite{Liu2024} are universal approximators in the classical sense, even when the spline degree and knot sequence are fixed in advance. This paper also has implications for the theory of superpositions of real functions. In particular, we show that every continuous multivariate function can be approximated arbitrarily well using only the coordinate functions and two fixed univariate functions under repeated addition and composition.