Convergence rates of the stochastic alternating algorithm for bi-objective optimization

TL;DR

This paper analyzes the convergence rates of stochastic alternating algorithms for bi-objective optimization, showing sublinear rates under strong convexity and extending results to convex and non-smooth cases, enabling Pareto front approximation.

Contribution

It provides the first convergence rate analysis for stochastic alternating algorithms in bi-objective optimization, including strong convexity, convexity, and non-smooth cases.

Findings

Achieves $ ext{O}(1/T)$ convergence rate under strong convexity.

Extends to $ ext{O}(1/ oot{T}{})$ rate in convex case.

Allows Pareto front approximation by adjusting step proportions.

Abstract

Stochastic alternating algorithms for bi-objective optimization are considered when optimizing two conflicting functions for which optimization steps have to be applied separately for each function. Such algorithms consist of applying a certain number of steps of gradient or subgradient descent on each single objective at each iteration. In this paper, we show that stochastic alternating algorithms achieve a sublinear convergence rate of $\mathcal{O}(1/T)$, under strong convexity, for the determination of a minimizer of a weighted-sum of the two functions, parameterized by the number of steps applied on each of them. An extension to the convex case is presented for which the rate weakens to $\mathcal{O}(1/\sqrt{T})$. These rates are valid also in the non-smooth case. Importantly, by varying the proportion of steps applied to each function, one can determine an approximation to the Pareto front.