The Neural Differential Manifold: An Architecture with Explicit Geometric Structure
Di Zhang

TL;DR
The Neural Differential Manifold (NDM) introduces a neural network architecture that explicitly incorporates geometric structures, such as Riemannian metrics, into its design for improved interpretability, regularization, and optimization.
Contribution
This work presents the NDM architecture that models neural networks as differentiable manifolds with explicit geometric structure, a novel approach in deep learning.
Findings
Enhanced interpretability of internal representations.
Improved generalization and robustness through geometric regularization.
Potential for more efficient optimization and continual learning.
Abstract
This paper introduces the Neural Differential Manifold (NDM), a novel neural network architecture that explicitly incorporates geometric structure into its fundamental design. Departing from conventional Euclidean parameter spaces, the NDM re-conceptualizes a neural network as a differentiable manifold where each layer functions as a local coordinate chart, and the network parameters directly parameterize a Riemannian metric tensor at every point. The architecture is organized into three synergistic layers: a Coordinate Layer implementing smooth chart transitions via invertible transformations inspired by normalizing flows, a Geometric Layer that dynamically generates the manifold's metric through auxiliary sub-networks, and an Evolution Layer that optimizes both task performance and geometric simplicity through a dual-objective loss function. This geometric regularization penalizes…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
