CHLU: The Causal Hamiltonian Learning Unit as a Symplectic Primitive for Deep Learning

TL;DR

The paper introduces the Causal Hamiltonian Learning Unit (CHLU), a physics-inspired deep learning primitive that balances stability and memory retention in temporal models by conserving phase-space volume through symplectic integration.

Contribution

It proposes a novel Hamiltonian-based primitive for deep learning that enforces a relativistic structure to improve stability and information preservation over time.

Findings

Demonstrated CHLU's generative ability on MNIST

Achieved infinite-horizon stability in temporal modeling

Conserved phase-space volume with symplectic integration

Abstract

Current deep learning primitives dealing with temporal dynamics suffer from a fundamental dichotomy: they are either discrete and unstable (LSTMs) \citep{pascanu_difficulty_2013}, leading to exploding or vanishing gradients; or they are continuous and dissipative (Neural ODEs) \citep{dupont_augmented_2019}, which destroy information over time to ensure stability. We propose the \textbf{Causal Hamiltonian Learning Unit} (pronounced: \textit{clue}), a novel Physics-grounded computational learning primitive. By enforcing a Relativistic Hamiltonian structure and utilizing symplectic integration, a CHLU strictly conserves phase-space volume, as an attempt to solve the memory-stability trade-off. We show that the CHLU is designed for infinite-horizon stability, as well as controllable noise filtering. We then demonstrate a CHLU's generative ability using the MNIST dataset as a proof-of-principle.