An Invariant Compiler for Neural ODEs in AI-Accelerated Scientific Simulation

TL;DR

This paper introduces an invariant compiler that enforces physical invariants in neural ODEs by construction, ensuring physically plausible solutions in scientific simulations and separating scientific structure from learned dynamics.

Contribution

It presents a novel compiler framework that guarantees invariants in neural ODEs by design, using an LLM-driven process to produce structure-preserving architectures.

Findings

Trajectories remain on the admissible manifold in continuous time.

The compiler enforces invariants by construction, not just regularization.

Provides a systematic design pattern for invariant-respecting neural surrogates.

Abstract

Neural ODEs are increasingly used as continuous-time models for scientific and sensor data, but unconstrained neural ODEs can drift and violate domain invariants (e.g., conservation laws), yielding physically implausible solutions. In turn, this can compound error in long-horizon prediction and surrogate simulation. Existing solutions typically aim to enforce invariance by soft penalties or other forms of regularization, which can reduce overall error but do not guarantee that trajectories will not leave the constraint manifold. We introduce the invariant compiler, a framework that enforces invariants by construction: it treats invariants as first-class types and uses an LLM-driven compilation workflow to translate a generic neural ODE specification into a structure-preserving architecture whose trajectories remain on the admissible manifold in continuous time (and up to numerical integration error in practice). This compiler view cleanly separates what must be preserved (scientific structure) from what is learned from data (dynamics within that structure). It provides a systematic design pattern for invariant-respecting neural surrogates across scientific domains.