Learning Latent Graph Geometry via Fixed-Point Schr\"odinger-Type Activation: A Theoretical Study

TL;DR

This paper introduces a theoretical framework for neural networks based on stationary states of Schrödinger-type dynamics on learned latent graphs, linking various architectures and analyzing their properties.

Contribution

It develops a novel theoretical approach connecting stationary graph-based neural architectures with global optimization and relaxation techniques.

Findings

Multilayer stationary networks are equivalent to global stationary problems on supra-graphs.

Penalized relaxations converge to exact solutions as penalty increases.

Structural identifications yield complexity bounds based on sparse graph geometry.

Abstract

We study neural architectures in which each hidden layer is defined by the stationary state of a dissipative Schr\"odinger-type dynamics on a learned latent graph. On stable branches, the local stationary problem defines a differentiable implicit graph layer. To learn the graph itself, we optimize over the stratified moduli space of weighted graphs and equip each stratum with a non-degenerate K\"ahler-Hessian metric that keeps natural-gradient descent and face crossing well posed. We then show that a multilayer stationary network is equivalent to an exact global stationary problem on a supra-graph, and that it admits a penalized global relaxation whose stationary states converge to the exact one as the penalty parameter tends to infinity. Reverse-mode differentiation is recovered as the adjoint of the exact global system, and the penalized adjoint converges to it in the same limit. Finally, under finite-dimensional strong-monotonicity and admissible-lift assumptions, the corresponding represented hypothesis classes coincide among resolvent feed-forward networks, graph-stationary networks, supra-graph stationary systems, and sheaf-based architectures with unitary connection. The resulting structural identifications yield complexity bounds controlled by sparse graph or supra-graph geometry rather than dense ambient connectivity.