Geometry-Preserving Neural Architectures on Manifolds with Boundary

TL;DR

This paper introduces geometry-preserving neural architectures on manifolds with boundaries, utilizing dynamical systems and heat kernel methods to ensure geometric fidelity and universal approximation capabilities.

Contribution

It proposes a unified framework for geometry-aware neural networks on manifolds, including new universal approximation results and methods for learning projections when constraints are unknown.

Findings

Universal approximation for constrained neural ODEs.

Effective learned projections via heat-kernel limits.

Strong experimental results on S^2, SO(3), and diffusion tasks.

Abstract

Preserving geometric structure is important in learning. We propose a unified class of geometry-aware architectures that interleave geometric updates between layers, where both projection layers and intrinsic exponential map updates arise as discretizations of projected dynamical systems on manifolds (with or without boundary). Within this framework, we establish universal approximation results for constrained neural ODEs. We also analyze architectures that enforce geometry only at the output, proving a separate universal approximation property that enables direct comparison to interleaved designs. When the constraint set is unknown, we learn projections via small-time heat-kernel limits, showing diffusion/flow-matching can be used as data-based projections. Experiments on dynamics over S^2 and SO(3), and diffusion on S^{d-1}-valued features demonstrate exact feasibility for analytic updates and strong performance for learned projections