Separable Physics-Informed Neural Networks

TL;DR

Separable PINNs (SPINN) introduce a novel architecture and training algorithm that significantly reduce computational costs and improve accuracy in solving complex multi-dimensional PDEs, including chaotic Navier-Stokes equations.

Contribution

The paper proposes a separable architecture and forward-mode automatic differentiation for PINNs, enabling efficient training on high-dimensional PDEs with over 10^7 collocation points.

Findings

Drastically reduced computational costs (62x wall-clock time, 1,394x FLOPs)

Faster solution of chaotic Navier-Stokes equations (9 min vs 10 hours)

Successful solution of 3+1 dimensional Navier-Stokes PDEs

Abstract

Physics-informed neural networks (PINNs) have recently emerged as promising data-driven PDE solvers showing encouraging results on various PDEs. However, there is a fundamental limitation of training PINNs to solve multi-dimensional PDEs and approximate highly complex solution functions. The number of training points (collocation points) required on these challenging PDEs grows substantially, but it is severely limited due to the expensive computational costs and heavy memory overhead. To overcome this issue, we propose a network architecture and training algorithm for PINNs. The proposed method, separable PINN (SPINN), operates on a per-axis basis to significantly reduce the number of network propagations in multi-dimensional PDEs unlike point-wise processing in conventional PINNs. We also propose using forward-mode automatic differentiation to reduce the computational cost of computing PDE residuals, enabling a large number of collocation points (>10^7) on a single commodity GPU. The experimental results show drastically reduced computational costs (62x in wall-clock time, 1,394x in FLOPs given the same number of collocation points) in multi-dimensional PDEs while achieving better accuracy. Furthermore, we present that SPINN can solve a chaotic (2+1)-d Navier-Stokes equation significantly faster than the best-performing prior method (9 minutes vs 10 hours in a single GPU), maintaining accuracy. Finally, we showcase that SPINN can accurately obtain the solution of a highly nonlinear and multi-dimensional PDE, a (3+1)-d Navier-Stokes equation. For visualized results and code, please see https://jwcho5576.github.io/spinn.github.io/.