Fast and scalable computation of shape-morphing nonlinear solutions with application to evolutional neural networks

TL;DR

This paper introduces three scalable methods for computing reduced-order nonlinear solutions (RONS) to PDEs, significantly improving speed and accuracy, and extends RONS application to neural network parameter evolution without training.

Contribution

The paper develops symbolic, collocation, and regularized RONS methods to overcome computational challenges in RONS, enabling its use for high-dimensional PDEs and neural network parameter evolution.

Findings

Achieved several orders of magnitude speedup and improved accuracy in examples

Extended RONS applicability to high-dimensional PDEs like Fokker-Planck and Kuramoto-Sivashinsky

Demonstrated RONS can optimize neural network parameters without training

Abstract

We develop fast and scalable methods for computing reduced-order nonlinear solutions (RONS). RONS was recently proposed as a framework for reduced-order modeling of time-dependent partial differential equations (PDEs), where the modes depend nonlinearly on a set of time-varying parameters. RONS uses a set of ordinary differential equations (ODEs) for the parameters to optimally evolve the shape of the modes to adapt to the PDE's solution. This method has already proven extremely effective in tackling challenging problems such as advection-dominated flows and high-dimensional PDEs. However, as the number of parameters grow, integrating the RONS equation and even its formation become computationally prohibitive. Here, we develop three separate methods to address these computational bottlenecks: symbolic RONS, collocation RONS and regularized RONS. We demonstrate the efficacy of these methods on two examples: Fokker-Planck equation in high dimensions and the Kuramoto-Sivashinsky equation. In both cases, we observe that the proposed methods lead to several orders of magnitude in speedup and accuracy. Our proposed methods extend the applicability of RONS beyond reduced-order modeling by making it possible to use RONS for accurate numerical solution of linear and nonlinear PDEs. Finally, as a special case of RONS, we discuss its application to problems where the PDE's solution is approximated by a neural network, with the time-dependent parameters being the weights and biases of the network. The RONS equations dictate the optimal evolution of the network's parameters without requiring any training.