Efficient Learning of PDEs via Taylor Expansion and Sparse Decomposition into Value and Fourier Domains

TL;DR

Reel is a novel method that accelerates PDE learning by decomposing updates into sparse components in value and frequency domains, using Taylor expansion and random projection for efficient, broad applicability.

Contribution

The paper introduces Reel, a new approach combining Taylor expansion and sparse decomposition in value and Fourier domains for faster PDE learning from data.

Findings

Achieves 70-98% reduction in training time with compressed data.

Maintains comparable model quality despite significant data compression.

Applicable to a broader class of PDEs than previous sparse methods.

Abstract

Accelerating the learning of Partial Differential Equations (PDEs) from experimental data will speed up the pace of scientific discovery. Previous randomized algorithms exploit sparsity in PDE updates for acceleration. However such methods are applicable to a limited class of decomposable PDEs, which have sparse features in the value domain. We propose Reel, which accelerates the learning of PDEs via random projection and has much broader applicability. Reel exploits the sparsity by decomposing dense updates into sparse ones in both the value and frequency domains. This decomposition enables efficient learning when the source of the updates consists of gradually changing terms across large areas (sparse in the frequency domain) in addition to a few rapid updates concentrated in a small set of "interfacial" regions (sparse in the value domain). Random projection is then applied to compress the sparse signals for learning. To expand the model applicability, Taylor series expansion is used in Reel to approximate the nonlinear PDE updates with polynomials in the decomposable form. Theoretically, we derive a constant factor approximation between the projected loss function and the original one with poly-logarithmic number of projected dimensions. Experimentally, we provide empirical evidence that our proposed Reel can lead to faster learning of PDE models (70-98% reduction in training time when the data is compressed to 1% of its original size) with comparable quality as the non-compressed models.