Functional Tensor Decompositions for Physics-Informed Neural Networks

TL;DR

This paper introduces tensor decomposition techniques into Physics-Informed Neural Networks to improve their ability to solve high-dimensional PDEs by separating variables and enhancing performance.

Contribution

It proposes a novel tensor decomposition-based framework for PINNs, enabling variable separation and improved approximation of multivariate functions in high dimensions.

Findings

Enhanced accuracy on complex high-dimensional PDEs

Effective variable separation using tensor decompositions

Surpassed state-of-the-art performance in PDE approximation

Abstract

Physics-Informed Neural Networks (PINNs) have shown continuous and increasing promise in approximating partial differential equations (PDEs), although they remain constrained by the curse of dimensionality. In this paper, we propose a generalized PINN version of the classical variable separable method. To do this, we first show that, using the universal approximation theorem, a multivariate function can be approximated by the outer product of neural networks, whose inputs are separated variables. We leverage tensor decomposition forms to separate the variables in a PINN setting. By employing Canonic Polyadic (CP), Tensor-Train (TT), and Tucker decomposition forms within the PINN framework, we create robust architectures for learning multivariate functions from separate neural networks connected by outer products. Our methodology significantly enhances the performance of PINNs, as evidenced by improved results on complex high-dimensional PDEs, including the 3d Helmholtz and 5d Poisson equations, among others. This research underscores the potential of tensor decomposition-based variably separated PINNs to surpass the state-of-the-art, offering a compelling solution to the dimensionality challenge in PDE approximation.