Vectorized Conditional Neural Fields: A Framework for Solving Time-dependent Parametric Partial Differential Equations

TL;DR

This paper introduces Vectorized Conditional Neural Fields (VCNeFs), a novel neural framework that efficiently solves time-dependent PDEs by modeling solutions as neural fields with attention mechanisms, enabling superior generalization and super-resolution.

Contribution

The paper presents VCNeFs, a new neural architecture that computes solutions for multiple spatio-temporal points in parallel and conditions on PDE parameters, addressing key limitations of existing models.

Findings

VCNeFs outperform existing ML surrogate models.

VCNeFs support zero-shot super-resolution in space and time.

VCNeFs generalize well to unseen PDE parameters.

Abstract

Transformer models are increasingly used for solving Partial Differential Equations (PDEs). Several adaptations have been proposed, all of which suffer from the typical problems of Transformers, such as quadratic memory and time complexity. Furthermore, all prevalent architectures for PDE solving lack at least one of several desirable properties of an ideal surrogate model, such as (i) generalization to PDE parameters not seen during training, (ii) spatial and temporal zero-shot super-resolution, (iii) continuous temporal extrapolation, (iv) support for 1D, 2D, and 3D PDEs, and (v) efficient inference for longer temporal rollouts. To address these limitations, we propose Vectorized Conditional Neural Fields (VCNeFs), which represent the solution of time-dependent PDEs as neural fields. Contrary to prior methods, however, VCNeFs compute, for a set of multiple spatio-temporal query points, their solutions in parallel and model their dependencies through attention mechanisms. Moreover, VCNeF can condition the neural field on both the initial conditions and the parameters of the PDEs. An extensive set of experiments demonstrates that VCNeFs are competitive with and often outperform existing ML-based surrogate models.