Field-Space Attention for Structure-Preserving Earth System Transformers
Maximilian Witte, Johannes Meuer, \'Etienne Pl\'esiat, Christopher Kadow

TL;DR
This paper introduces Field-Space attention, a novel Earth system Transformer mechanism that operates directly on physical fields to enhance interpretability, stability, and physical consistency in Earth system modeling.
Contribution
It presents a new attention mechanism that computes in the physical domain, preserving geometric structure and enabling better integration of multiscale Earth system data.
Findings
Faster and more stable convergence than standard Vision Transformers.
Requires fewer parameters while maintaining high fidelity.
Enables embedding of physical priors directly into the model.
Abstract
Accurate and physically consistent modeling of Earth system dynamics requires machine-learning architectures that operate directly on continuous geophysical fields and preserve their underlying geometric structure. Here we introduce Field-Space attention, a mechanism for Earth system Transformers that computes attention in the physical domain rather than in a learned latent space. By maintaining all intermediate representations as continuous fields on the sphere, the architecture enables interpretable internal states and facilitates the enforcement of scientific constraints. The model employs a fixed, non-learned multiscale decomposition and learns structure-preserving deformations of the input field, allowing coherent integration of coarse and fine-scale information while avoiding the optimization instabilities characteristic of standard single-scale Vision Transformers. Applied to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks · Neural Networks and Reservoir Computing · Geomagnetism and Paleomagnetism Studies
