A Kolmogorov High Order Deep Neural Network for High Frequency Partial Differential Equations in High Dimensions

TL;DR

This paper introduces a Kolmogorov high order deep neural network (K-HOrderDNN) that efficiently solves high-dimensional, high-frequency PDEs by mitigating the curse of dimensionality through a superposition-based approach, demonstrating superior accuracy and efficiency.

Contribution

The paper proposes K-HOrderDNN, leveraging Kolmogorov superposition theorem to reduce basis functions and improve high-dimensional PDE solving, outperforming traditional high order DNNs.

Findings

K-HOrderDNN reduces basis functions from exponential to linear in dimension.

K-HOrderDNN achieves two orders of magnitude lower error in high-dimensional problems.

K-HOrderDNN demonstrates faster convergence and higher accuracy for high-frequency PDEs.

Abstract

This paper proposes a Kolmogorov high order deep neural network (K-HOrderDNN) for solving high-dimensional partial differential equations (PDEs), which improves the high order deep neural networks (HOrderDNNs). HOrderDNNs have been demonstrated to outperform conventional DNNs for high frequency problems by introducing a nonlinear transformation layer consisting of $(p+1)^d$ basis functions. However, the number of basis functions grows exponentially with the dimension $d$, which results in the curse of dimensionality (CoD). Inspired by the Kolmogorov superposition theorem (KST), which expresses a multivariate function as superpositions of univariate functions and addition, K-HOrderDNN utilizes a HOrderDNN to efficiently approximate univariate inner functions instead of directly approximating the multivariate function, reducing the number of introduced basis functions to $d(p+1)$. We theoretically demonstrate that CoD is mitigated when target functions belong to a dense subset of continuous multivariate functions. Extensive numerical experiments show that: for high-dimensional problems ($d$=10, 20, 50) where HOrderDNNs($p>1$) are intractable, K-HOrderDNNs($p>1$) exhibit remarkable performance. Specifically, when $d=10$, K-HOrderDNN($p=7$) achieves an error of 4.40E-03, two orders of magnitude lower than that of HOrderDNN($p=1$) (see Table 10); for high frequency problems, K-HOrderDNNs($p>1$) can achieve higher accuracy with fewer parameters and faster convergence rates compared to HOrderDNNs (see Table 8).