Physical Transformer
Tao Xu, Zhixin Hu, Li Luo, Momiao Xiong

TL;DR
This paper introduces a physical transformer model that integrates geometric representation and physical dynamics into transformer architectures, enabling more interpretable and physically grounded AI reasoning.
Contribution
It proposes a novel hierarchical framework combining transformer computation with Hamiltonian dynamics and geometric invariants, bridging digital AI and physical systems.
Findings
Outperforms naive baselines in stability and accuracy on toy dynamical problems.
Maintains geometric and energetic invariants through symplectic discretization.
Provides a pathway for physically grounded, interpretable AI models.
Abstract
Digital AI systems spanning large language models, vision models, and generative architectures that operate primarily in symbolic, linguistic, or pixel domains. They have achieved striking progress, but almost all of this progress lives in virtual spaces. These systems transform embeddings and tokens, yet do not themselves touch the world and rarely admit a physical interpretation. In this work we propose a physical transformer that couples modern transformer style computation with geometric representation and physical dynamics. At the micro level, attention heads, and feed-forward blocks are modeled as interacting spins governed by effective Hamiltonians plus non-Hamiltonian bath terms. At the meso level, their aggregated state evolves on a learned Neural Differential Manifold (NDM) under Hamiltonian flows and Hamilton, Jacobi, Bellman (HJB) optimal control, discretized by symplectic…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFerroelectric and Negative Capacitance Devices · Generative Adversarial Networks and Image Synthesis · Multimodal Machine Learning Applications
