Solving parametric partial differential equations with deep rectified quadratic unit neural networks

TL;DR

This paper demonstrates that deep rectified quadratic unit neural networks can efficiently approximate solution maps of parametric PDEs, with improved complexity bounds over ReLU networks, leveraging low-dimensional solution manifolds.

Contribution

It provides a new theoretical complexity bound for deep ReQU neural networks in solving parametric PDEs, showing improved efficiency over ReLU networks.

Findings

ReQU networks achieve an upper bound complexity of $ ilde{O}(d^3 ext{log}^{q} ext{log}(1/\epsilon))$

Numerical experiments verify the theoretical complexity bounds

ReQU networks better exploit low-dimensional solution manifolds

Abstract

Implementing deep neural networks for learning the solution maps of parametric partial differential equations (PDEs) turns out to be more efficient than using many conventional numerical methods. However, limited theoretical analyses have been conducted on this approach. In this study, we investigate the expressive power of deep rectified quadratic unit (ReQU) neural networks for approximating the solution maps of parametric PDEs. The proposed approach is motivated by the recent important work of G. Kutyniok, P. Petersen, M. Raslan and R. Schneider (Gitta Kutyniok, Philipp Petersen, Mones Raslan, and Reinhold Schneider. A theoretical analysis of deep neural networks and parametric pdes. Constructive Approximation, pages 1-53, 2021), which uses deep rectified linear unit (ReLU) neural networks for solving parametric PDEs. In contrast to the previously established complexity-bound $\mathcal{O}\left(d^3\log_{2}^{q}(1/ \epsilon) \right)$ for ReLU neural networks, we derive an upper bound $\mathcal{O}\left(d^3\log_{2}^{q}\log_{2}(1/ \epsilon) \right)$ on the size of the deep ReQU neural network required to achieve accuracy $\epsilon>0$, where $d$ is the dimension of reduced basis representing the solutions. Our method takes full advantage of the inherent low-dimensionality of the solution manifolds and better approximation performance of deep ReQU neural networks. Numerical experiments are performed to verify our theoretical result.