Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications

TL;DR

This paper develops a tighter single-timescale analysis for multi-sequence stochastic approximation (MSSA) without requiring fixed point smoothness, showing convergence rates under strong monotonicity and applying results to bilevel optimization and distributed learning.

Contribution

It provides the first single-timescale convergence analysis for MSSA without fixed point smoothness assumptions, matching single-sequence SA rates.

Findings

MSSA converges at rate  (K^{-1}) when operators are strongly monotone.

MSSA converges at rate  (K^{-1/2}) with all but the main operator strongly monotone.

Applications to bilevel optimization and distributed learning demonstrate practical benefits.

Abstract

Stochastic approximation (SA) that involves multiple coupled sequences, known as multiple-sequence SA (MSSA), finds diverse applications in the fields of signal processing and machine learning. However, existing theoretical understandings {of} MSSA are limited: the multi-timescale analysis implies a slow convergence rate, whereas the single-timescale analysis relies on a stringent fixed point smoothness assumption. This paper establishes tighter single-timescale analysis for MSSA, without assuming smoothness of the fixed points. Our theoretical findings reveal that, when all involved operators are strongly monotone, MSSA converges at a rate of $\tilde{\mathcal{O}}(K^{-1})$, where $K$ denotes the total number of iterations. In addition, when all involved operators are strongly monotone except for the main one, MSSA converges at a rate of $\mathcal{O}(K^{-\frac{1}{2}})$. These theoretical findings align with those established for single-sequence SA. Applying these theoretical findings to bilevel optimization and communication-efficient distributed learning offers relaxed assumptions and/or simpler algorithms with performance guarantees, as validated by numerical experiments.