Discovering Physical Directions in Weight Space: Composing Neural PDE Experts
Pengkai Wang, Pengwei Liu, Yuanyi Wang, Guanyu Chen, Xingyu Ren, Xiaolong Li, Zhongkai Hao, Yuting Kong, Qixin Zhang, Dong Ni

TL;DR
This paper introduces a method to identify and utilize physical directions in neural network weight space for PDE modeling, enabling effective transfer and composition of neural PDE experts across different physical regimes.
Contribution
It proposes a novel perspective interpreting endpoint experts as probes of physical directions and introduces CCM, a method for composing neural PDE experts along these directions.
Findings
CCM reduces out-of-distribution error by up to 54.2%
Endpoint fine-tuning reveals a calibratable physical direction
Experiments confirm physical interpretability of weight-space directions
Abstract
Recent advances in neural operators have made partial differential equation (PDE) surrogate modeling increasingly scalable and transferable through large-scale pretraining and in-context adaptation. However, after a shared operator is fine-tuned to multiple regimes within a continuous physical family, it remains unclear whether the resulting weight-space updates merely form isolated regime experts or reveal reusable physical structure. Starting from a shared family anchor, we fine-tune low- and high-regime endpoint experts and show that their updates can be separated into a family-shared adaptation and a direction aligned with the underlying physical parameter. This separation reinterprets endpoint experts as finite-difference probes of a local physical direction in weight space, explaining why static averaging can interpolate between regimes but attenuates endpoint-specific physics.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
