DOF: Accelerating High-order Differential Operators with Forward Propagation

TL;DR

The paper introduces DOF, a novel computational framework that accelerates the calculation of high-order differential operators in PDE solving, offering significant efficiency and memory improvements over existing methods.

Contribution

It presents a new forward-propagation-based method for computing second-order derivatives that outperforms traditional automatic differentiation in speed and memory usage.

Findings

Achieves 2x speedup over AutoDiff on MLPs.

Nearly 20x speedup on sparse Jacobian MLPs.

Reduces memory consumption significantly.

Abstract

Solving partial differential equations (PDEs) efficiently is essential for analyzing complex physical systems. Recent advancements in leveraging deep learning for solving PDE have shown significant promise. However, machine learning methods, such as Physics-Informed Neural Networks (PINN), face challenges in handling high-order derivatives of neural network-parameterized functions. Inspired by Forward Laplacian, a recent method of accelerating Laplacian computation, we propose an efficient computational framework, Differential Operator with Forward-propagation (DOF), for calculating general second-order differential operators without losing any precision. We provide rigorous proof of the advantages of our method over existing methods, demonstrating two times improvement in efficiency and reduced memory consumption on any architectures. Empirical results illustrate that our method surpasses traditional automatic differentiation (AutoDiff) techniques, achieving 2x improvement on the MLP structure and nearly 20x improvement on the MLP with Jacobian sparsity.