Long term dynamics of the subgradient method for Lipschitz path differentiable functions

TL;DR

This paper investigates the long-term behavior of the subgradient method with diminishing stepsizes on non-smooth, non-convex functions that are locally Lipschitz and path differentiable, focusing on oscillation dynamics and convergence properties.

Contribution

It introduces a novel analysis of oscillation behavior using closed measures, extending convergence results and revealing a local oscillation compensation principle.

Findings

Time averages of gradients around limit points tend to vanish.

Oscillations are perpendicular to the general drift.

Established new convergence and oscillation structure results.

Abstract

We consider the long-term dynamics of the vanishing stepsize subgradient method in the case when the objective function is neither smooth nor convex. We assume that this function is locally Lipschitz and path differentiable, i.e., admits a chain rule. Our study departs from other works in the sense that we focus on the behavoir of the oscillations, and to do this we use closed measures. We recover known convergence results, establish new ones, and show a local principle of oscillation compensation for the velocities. Roughly speaking, the time average of gradients around one limit point vanishes. This allows us to further analyze the structure of oscillations, and establish their perpendicularity to the general drift.