FastLRNR and Sparse Physics Informed Backpropagation

TL;DR

This paper presents SPInProp, a method that accelerates backpropagation in Low Rank Neural Representation networks by exploiting their low rank structure, leading to faster solutions of parametrized PDEs in physics-informed neural networks.

Contribution

Introduction of SPInProp, a novel approach that reduces neural network complexity by leveraging low rank structures, enabling faster physics-informed neural network training.

Findings

Significant reduction in backpropagation complexity.

Accelerated solutions for parametrized PDEs.

Effective application within physics-informed neural networks.

Abstract

We introduce Sparse Physics Informed Backpropagation (SPInProp), a new class of methods for accelerating backpropagation for a specialized neural network architecture called Low Rank Neural Representation (LRNR). The approach exploits the low rank structure within LRNR and constructs a reduced neural network approximation that is much smaller in size. We call the smaller network FastLRNR. We show that backpropagation of FastLRNR can be substituted for that of LRNR, enabling a significant reduction in complexity. We apply SPInProp to a physics informed neural networks framework and demonstrate how the solution of parametrized partial differential equations is accelerated.