CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations

TL;DR

CoLoRA introduces a continuous low-rank adaptation method for neural networks that efficiently models the evolution of solutions to parameterized PDEs, reducing training data needs and significantly speeding up predictions.

Contribution

The paper proposes a novel continuous low-rank adaptation technique for neural PDE models that enhances efficiency and accuracy in data-scarce scenarios.

Findings

Predictions are orders of magnitude faster than classical methods.

High accuracy and parameter efficiency compared to other neural approaches.

Requires fewer offline training trajectories due to local time approximation.

Abstract

This work introduces reduced models based on Continuous Low Rank Adaptation (CoLoRA) that pre-train neural networks for a given partial differential equation and then continuously adapt low-rank weights in time to rapidly predict the evolution of solution fields at new physics parameters and new initial conditions. The adaptation can be either purely data-driven or via an equation-driven variational approach that provides Galerkin-optimal approximations. Because CoLoRA approximates solution fields locally in time, the rank of the weights can be kept small, which means that only few training trajectories are required offline so that CoLoRA is well suited for data-scarce regimes. Predictions with CoLoRA are orders of magnitude faster than with classical methods and their accuracy and parameter efficiency is higher compared to other neural network approaches.