Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation

TL;DR

The paper introduces Chebyshev Kolmogorov-Arnold Networks, a novel neural architecture leveraging Chebyshev polynomials inspired by the Kolmogorov-Arnold theorem, to improve nonlinear function approximation efficiency and interpretability.

Contribution

It proposes a new neural network architecture that integrates Chebyshev polynomials for enhanced approximation capabilities, inspired by the Kolmogorov-Arnold theorem, and demonstrates its effectiveness over traditional MLPs.

Findings

Chebyshev KANs outperform MLPs in parameter efficiency.

They show improved interpretability in function approximation.

Effective in digit classification and fractal function generation.

Abstract

Accurate approximation of complex nonlinear functions is a fundamental challenge across many scientific and engineering domains. Traditional neural network architectures, such as Multi-Layer Perceptrons (MLPs), often struggle to efficiently capture intricate patterns and irregularities present in high-dimensional functions. This paper presents the Chebyshev Kolmogorov-Arnold Network (Chebyshev KAN), a new neural network architecture inspired by the Kolmogorov-Arnold representation theorem, incorporating the powerful approximation capabilities of Chebyshev polynomials. By utilizing learnable functions parametrized by Chebyshev polynomials on the network's edges, Chebyshev KANs enhance flexibility, efficiency, and interpretability in function approximation tasks. We demonstrate the efficacy of Chebyshev KANs through experiments on digit classification, synthetic function approximation, and fractal function generation, highlighting their superiority over traditional MLPs in terms of parameter efficiency and interpretability. Our comprehensive evaluation, including ablation studies, confirms the potential of Chebyshev KANs to address longstanding challenges in nonlinear function approximation, paving the way for further advancements in various scientific and engineering applications.