The Propagation Field: A Geometric Substrate Theory of Deep Learning

TL;DR

This paper introduces the concept of a neural propagation field to understand deep learning models through their internal geometry, revealing new insights into model behavior and robustness.

Contribution

It defines the propagation field as hidden-state trajectories and Jacobians, showing how endpoint training constrains only boundary behavior, and proposes metrics and regularization to improve model properties.

Findings

Endpoint-equivalent models can differ greatly in internal structure.

Field-aware objectives enhance generalization, robustness, and calibration.

Field-preservation regularization improves continual learning metrics.

Abstract

Modern deep learning treats neural networks primarily as endpoint functions from inputs to outputs. Inspired by the shift from force to geometry in physics, we ask whether a network should instead be understood through the geometry of its internal propagation. We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth. Endpoint losses constrain only the boundary behavior of this field, leaving its interior geometry underdetermined. We show that endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure, and introduce observable field metrics such as path sensitivity, solver consistency, and trajectory/Jacobian retention. In controlled teacher-flow and PDE systems, endpoint fitting fails to recover the underlying propagation law. In real multi-path tasks, field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure, but can collapse when over-constrained. In continual learning, field-preservation regularization complements replay and distillation: on Split CIFAR-100, DER++ with field preservation improves average accuracy, backward transfer, and field-retention metrics. These results identify propagation-field quality as a measurable and trainable property of neural networks beyond endpoint performance.