Non-autoregressive time-series methods for stable parametric reduced-order models

TL;DR

This paper introduces a non-autoregressive time series method that enhances the stability and speed of machine learning-based reduced-order models for advection-dominated systems, demonstrated on shallow water equations.

Contribution

The authors develop a non-autoregressive LSTM variant that improves stability and inference speed of reduced-order models for PDEs, addressing stability issues in traditional autoregressive approaches.

Findings

Non-autoregressive LSTM improves stability over autoregressive models.

Inference times are reduced by three orders of magnitude.

Bidirectional PCA-based approach yields best accuracy.

Abstract

Advection-dominated dynamical systems, characterized by partial differential equations, are found in applications ranging from weather forecasting to engineering design where accuracy and robustness are crucial. There has been significant interest in the use of techniques borrowed from machine learning to reduce the computational expense and/or improve the accuracy of predictions for these systems. These rely on the identification of a basis that reduces the dimensionality of the problem and the subsequent use of time series and sequential learning methods to forecast the evolution of the reduced state. Often, however, machine-learned predictions after reduced-basis projection are plagued by issues of stability stemming from incomplete capture of multiscale processes as well as due to error growth for long forecast durations. To address these issues, we have developed a \emph{non-autoregressive} time series approach for predicting linear reduced-basis time histories of forward models. In particular, we demonstrate that non-autoregressive counterparts of sequential learning methods such as long short-term memory (LSTM) considerably improve the stability of machine-learned reduced-order models. We evaluate our approach on the inviscid shallow water equations and show that a non-autoregressive variant of the standard LSTM approach that is bidirectional in the PCA components obtains the best accuracy for recreating the nonlinear dynamics of partial observations. Moreover---and critical for many applications of these surrogates---inference times are reduced by three orders of magnitude using our approach, compared with both the equation-based Galerkin projection method and the standard LSTM approach.