First Order Algorithm on an Optimization Problem with Improved Convergence when Problem is Convex

TL;DR

This paper introduces a modified FISTA algorithm for optimization problems with a combination of convex and nonconvex functions, achieving improved convergence rates when the objective is convex.

Contribution

A novel first order algorithm that enhances convergence rates for convex problems compared to existing methods.

Findings

Iteration complexity of O(ε^{-2}) for general problems.

Improved complexity of O(ε^{-2/3}) when the objective is convex.

Asymptotic convergence rates with worst-case o(ε^{-2}) iterations.

Abstract

We propose a first order algorithm, a modified version of FISTA, to solve an optimization problem with an objective function that is a sum of a possibly nonconvex function, with Lipschitz continuous gradient, and a convex function which can be nonsmooth. The algorithm is shown to have an iteration complexity of $\mathcal{O}(\epsilon^{-2})$ to find an $\epsilon$-approximate solution to the problem, and this complexity improves to $\mathcal{O}(\epsilon^{-2/3})$ when the objective function turns out to be convex. We further provide asymptotic convergence rate for the algorithm of worst case $o(\epsilon^{-2})$ iterations to find an $\epsilon$-approximate solution to the problem, with worst case $o(\epsilon^{-2/3})$ iterations when its objective function is convex.