Continuous Limits of Coupled Flows in Representation Learning

TL;DR

This paper develops a theoretical framework connecting decentralized representation learning algorithms with continuous stochastic differential equations, proving convergence and feature disentanglement on data manifolds.

Contribution

It formalizes the convergence of discrete decentralized learning algorithms to continuous stochastic dynamics and shows emergent feature disentanglement.

Findings

Discrete spatial transitions converge to overdamped Langevin SDEs

Representation weights avoid divergence and align with principal eigenspace

Orthogonally disentangled features emerge at the stationary limit

Abstract

While modern representation learning relies heavily on global error signals, decentralized algorithms driven by local interactions offer a fundamental distributed alternative. However, the macroscopic convergence properties of these discrete dynamics on continuous data manifolds remain theoretically unresolved, notoriously suffering from parameter explosion. We bridge this gap by formalizing decentralized learning as a coupled slow-fast dynamical system on Riemannian manifolds. First, using measure-theoretic limits, we prove that the discrete spatial transitions converge uniformly to an overdamped Langevin stochastic differential equation. Second, via the It\^o-Poisson resolvent and a stochastic extension of LaSalle's Invariance Principle, we establish that the representation weights unconditionally avoid divergence and align strictly with the principal eigenspace of the spatial measure. Finally, we construct a joint Lyapunov functional for the fully coupled spatial-parametric flow. This proves global dissipativity and demonstrates that orthogonally disentangled, linearly separable features emerge spontaneously at the stationary limit. Our framework bridges discrete algorithms with continuous stochastic analysis, providing a formal theoretical baseline for decentralized representation learning.