Reduced Order Modeling using Shallow ReLU Networks with Grassmann Layers

TL;DR

This paper introduces a structured neural network approach for nonlinear model reduction that leverages Grassmann manifolds to determine reduced bases, improving efficiency and effectiveness especially in data-scarce scientific applications.

Contribution

The paper proposes a novel neural network architecture with a Grassmann manifold layer for reduced basis determination, enhancing nonlinear model reduction.

Findings

Effective in data-scarce regimes

Reduces network parameters for low-dimensional data

Applicable to nonlinear dynamical systems and aerospace problems

Abstract

This paper presents a nonlinear model reduction method for systems of equations using a structured neural network. The neural network takes the form of a "three-layer" network with the first layer constrained to lie on the Grassmann manifold and the first activation function set to identity, while the remaining network is a standard two-layer ReLU neural network. The Grassmann layer determines the reduced basis for the input space, while the remaining layers approximate the nonlinear input-output system. The training alternates between learning the reduced basis and the nonlinear approximation, and is shown to be more effective than fixing the reduced basis and training the network only. An additional benefit of this approach is, for data that lie on low-dimensional subspaces, that the number of parameters in the network does not need to be large. We show that our method can be applied to scientific problems in the data-scarce regime, which is typically not well-suited for neural network approximations. Examples include reduced order modeling for nonlinear dynamical systems and several aerospace engineering problems.