Time-Varying Convex Optimization via Time-Varying Averaged Operators

TL;DR

This paper introduces a new framework for time-varying convex optimization that removes strong convexity and smoothness assumptions, enabling more flexible and efficient algorithms for dynamic problems.

Contribution

It develops a comprehensive framework using averaged operator theory to analyze and improve running algorithms for time-varying convex optimization without restrictive assumptions.

Findings

New convergence results for running algorithms

Framework handles complex constraints like -regularization

Lifts previous limitations on convexity and smoothness

Abstract

Devising efficient algorithms that track the optimizers of continuously varying convex optimization problems is key in many applications. A possible strategy is to sample the time-varying problem at constant rate and solve the resulting time-invariant problem. This can be too computationally burdensome in many scenarios. An alternative strategy is to set up an iterative algorithm that generates a sequence of approximate optimizers, which are refined every time a new sampled time-invariant problem is available by one iteration of the algorithm. These types of algorithms are called running. A major limitation of current running algorithms is their key assumption of strong convexity and strong smoothness of the time-varying convex function. In addition, constraints are only handled in simple cases. This limits the current capability for running algorithms to tackle relevant problems, such as $\ell_1$-regularized optimization programs. In this paper, these assumptions are lifted by leveraging averaged operator theory and a fairly comprehensive framework for time-varying convex optimization is presented. In doing so, new results characterizing the convergence of running versions of a number of widely used algorithms are derived.