Inexact Proximal-Gradient Methods with Support Identification

TL;DR

This paper analyzes inexact proximal-gradient methods for composite convex optimization, establishing iteration complexity and support identification properties, especially when the proximal operator lacks a closed-form solution.

Contribution

It introduces adaptive termination conditions for inexact solutions and proves support identification for overlapping group l1 regularization.

Findings

Iteration complexity is O(τ^{-2}) for approximate stationary points.

Proposed algorithm asymptotically identifies the support of optimal solutions.

Provides bounds on iterations needed for support recovery under accuracy control.

Abstract

We consider the proximal-gradient method for minimizing an objective function that is the sum of a smooth function and a non-smooth convex function. A feature that distinguishes our work from most in the literature is that we assume that the associated proximal operator does not admit a closed-form solution. To address this challenge, we study two adaptive and implementable termination conditions that dictate how accurately the proximal-gradient subproblem is solved. We prove that the number of iterations required for the inexact proximal-gradient method to reach a $\tau > 0$ approximate first-order stationary point is $\mathcal{O}(\tau^{-2})$, which matches the similar result that holds when exact subproblem solutions are computed. Also, by focusing on the overlapping group $\ell_1$ regularizer, we propose an algorithm for approximately solving the proximal-gradient subproblem, and then prove that its iterates identify (asymptotically) the support of an optimal solution. If one imposes additional control over the accuracy to which each subproblem is solved, we give an upper bound on the maximum number of iterations before the support of an optimal solution is obtained.