Comparing different subgradient methods for solving convex optimization problems with functional constraints

TL;DR

This paper compares two reformulated subgradient methods for convex optimization with constraints, analyzing their complexity and performance through theoretical bounds and test examples.

Contribution

It introduces and compares two subgradient methods based on existing frameworks, highlighting their complexity differences and practical performance.

Findings

The Metel–Takeda-based method has complexity $ ilde{O}(rac{1}{ ext{epsilon}^{2r}})$ for all r>1.

Boyd's method has complexity $O(rac{1}{ ext{epsilon}^2})$.

Test examples demonstrate differences in efficiency and accuracy between the methods.

Abstract

We consider the problem of minimizing a convex, nonsmooth function subject to a closed convex constraint domain. The methods that we propose are reforms of subgradient methods based on Metel--Takeda's paper [Optimization Letters 15.4 (2021): 1491-1504] and Boyd's works [Lecture notes of EE364b, Stanford University, Spring 2013-14, pp. 1-39]. While the former has complexity $\mathcal{O}(\varepsilon^{-2r})$ for all $r> 1$, the complexity of the latter is $\mathcal{O}(\varepsilon^{-2})$. We perform some comparisons between these two methods using several test examples.