KAN-CL: Per-Knot Importance Regularization for Continual Learning with Kolmogorov-Arnold Networks

TL;DR

This paper introduces KAN-CL, a continual learning method that uses Kolmogorov-Arnold Networks' local parameterization to significantly reduce catastrophic forgetting while maintaining accuracy.

Contribution

It presents a novel importance-weighted regularization approach at per-knot granularity using KAN's spline parameterization, improving continual learning performance.

Findings

KAN-CL reduces forgetting by up to 93% on Split-CIFAR benchmarks.

It matches or exceeds baseline accuracy on multiple continual learning tasks.

NTK analysis shows KAN's local parameterization induces a structural rank deficit, aiding forgetting bounds.

Abstract

Catastrophic forgetting remains the central obstacle in continual learning (CL): parameters shared across tasks interfere with one another, and existing regularization methods such as EWC and SI apply uniform penalties without awareness of which input region a parameter serves. We propose KAN-CL, a continual learning framework that exploits the compact-support spline parameterization of Kolmogorov-Arnold Networks (KANs) to perform importance-weighted anchoring at per-knot granularity. Deployed as a classification head on a convolutional backbone with standard EWC regularization on the backbone (bbEWC) KAN-CL achieves forgetting reductions of 88% and 93% over a head-only KAN baseline on Split-CIFAR-10/5T and Split-CIFAR-100/10T respectively, while matching or exceeding the accuracy of all baselines on both benchmarks. We further provide a Neural Tangent Kernel (NTK) analysis showing that KAN's spline locality induces a structural rank deficit in the cross-task NTK, yielding a forgetting bound that holds even in the feature-learning regime. These results establish that combining an architecture with natural parameter locality (KAN head) with a complementary backbone regularizer (bbEWC) yields a compositional and principled approach to catastrophic forgetting.