Overtone: Cyclic Patch Modulation for Clean, Efficient, and Flexible Physics Emulators

TL;DR

Overtone introduces cyclic patch modulation for PDE surrogates, reducing harmonic errors and enabling flexible, compute-adaptive inference, achieving up to 40% lower long-term error compared to static patch models.

Contribution

It presents a novel cyclic patch modulation technique with dynamic patch size control, improving accuracy and flexibility of physics emulators.

Findings

Up to 40% reduction in long rollout error.

Matches or exceeds fixed-patch baselines across benchmarks.

Enables dynamic trade-offs between accuracy and speed.

Abstract

Transformer-based PDE surrogates achieve remarkable performance but face two key challenges: fixed patch sizes cause systematic error accumulation at harmonic frequencies, and computational costs remain inflexible regardless of problem complexity or available resources. We introduce Overtone, a unified solution through dynamic patch size control at inference. Overtone's key insight is that cyclically modulating patch sizes during autoregressive rollouts distributes errors across the frequency spectrum, mitigating the systematic harmonic artifact accumulation that plague fixed-patch models. We implement this through two architecture-agnostic modules--CSM (using dynamic stride modulation) and CKM (using dynamic kernel resizing)--that together provide both harmonic mitigation and compute-adaptive deployment. This flexible tokenization lets users trade accuracy for speed dynamically based on computational constraints, and the cyclic rollout strategy yields up to 40% lower long rollout error in variance-normalised RMSE (VRMSE) compared to conventional, static-patch surrogates. Across challenging 2D and 3D PDE benchmarks, one Overtone model matches or exceeds fixed-patch baselines across inference compute budgets, when trained under a fixed total training budget setting.