Analytic and Variational Stability in Deep Learning Systems

TL;DR

This paper introduces a unified analytic and variational framework for analyzing stability in deep learning systems, linking sensitivities, spectral norms, and learning parameters through a Lyapunov-based approach.

Contribution

It develops the Learning Stability Profile and the Fundamental Analytic Stability Theorem, providing a comprehensive stability analysis applicable to both smooth and non-smooth deep learning models.

Findings

Classical spectral stability results for feedforward networks

CFL-type conditions for residual architectures

Stability laws for stochastic gradient methods

Abstract

We propose a unified analytic and variational framework for stability in deep learning systems viewed as coupled representation-parameter dynamics. The central object is the Learning Stability Profile, which measures how infinitesimal perturbations propagate through representations, parameters, and update mechanisms along the learning trajectory. Our main result, the Fundamental Analytic Stability Theorem, shows that uniform boundedness of these sensitivities is equivalent, up to norm equivalence, to the existence of a Lyapunov-type energy dissipating along the learning flow. In smooth regimes, this yields explicit stability exponents linking spectral norms, activation regularity, step sizes, and learning rates to contractive behavior. Classical spectral stability of feedforward networks, CFL-type conditions for residual architectures, and temporal stability laws for stochastic gradient methods follow as direct consequences. The framework extends to non-smooth systems, including ReLU networks, proximal and projected updates, and stochastic subgradient flows, by replacing classical derivatives with Clarke generalized derivatives and smooth energies with variational Lyapunov functionals. The resulting theory provides a unified dynamical description of stability across architectures and optimization methods, clarifying how design and training choices jointly control robustness and sensitivity to perturbations.