Chebyshev Feature Neural Network for Accurate Function Approximation

TL;DR

This paper introduces the Chebyshev Feature Neural Network (CFNN), a novel deep learning architecture that employs learnable Chebyshev functions to achieve near machine accuracy in function approximation, demonstrating scalability up to 20 dimensions.

Contribution

The paper proposes a new neural network structure with learnable Chebyshev functions and a multi-stage training strategy, enabling highly accurate function approximation.

Findings

Achieves near machine accuracy in function approximation.

Effective for high-dimensional problems up to 20 dimensions.

Demonstrates scalability and robustness of the method.

Abstract

We present a new Deep Neural Network (DNN) architecture capable of approximating functions up to machine accuracy. Termed Chebyshev Feature Neural Network (CFNN), the new structure employs Chebyshev functions with learnable frequencies as the first hidden layer, followed by the standard fully connected hidden layers. The learnable frequencies of the Chebyshev layer are initialized with exponential distributions to cover a wide range of frequencies. Combined with a multi-stage training strategy, we demonstrate that this CFNN structure can achieve machine accuracy during training. A comprehensive set of numerical examples for dimensions up to $20$ are provided to demonstrate the effectiveness and scalability of the method.