The Curvature Rate {\lambda}: A Scalar Measure of Input-Space Sharpness in Neural Networks

TL;DR

The paper introduces the curvature rate {}, a new scalar measure of input-space sharpness for neural networks, which is interpretable, efficient, and correlates with generalization and robustness.

Contribution

It proposes a novel input-space curvature measure {} based on derivatives, unifying classical concepts and extending to neural networks, with a simple regularizer to control sharpness.

Findings

{} tracks high-frequency structure emergence during training.

CRR regularizer shapes {} and improves confidence calibration.

{} evolves predictably and correlates with model smoothness.

Abstract

Curvature influences generalization, robustness, and how reliably neural networks respond to small input perturbations. Existing sharpness metrics are typically defined in parameter space (e.g., Hessian eigenvalues) and can be expensive, sensitive to reparameterization, and difficult to interpret in functional terms. We introduce a scalar curvature measure defined directly in input space: the curvature rate {\lambda}, given by the exponential growth rate of higher-order input derivatives. Empirically, {\lambda} is estimated as the slope of log ||D^n f|| versus n for small n. This growth-rate perspective unifies classical analytic quantities: for analytic functions, {\lambda} corresponds to the inverse radius of convergence, and for bandlimited signals, it reflects the spectral cutoff. The same principle extends to neural networks, where {\lambda} tracks the emergence of high-frequency structure in the decision boundary. Experiments on analytic functions and neural networks (Two Moons and MNIST) show that {\lambda} evolves predictably during training and can be directly shaped using a simple derivative-based regularizer, Curvature Rate Regularization (CRR). Compared to Sharpness-Aware Minimization (SAM), CRR achieves similar accuracy while yielding flatter input-space geometry and improved confidence calibration. By grounding curvature in differentiation dynamics, {\lambda} provides a compact, interpretable, and parameterization-invariant descriptor of functional smoothness in learned models.