An Inexact Modified Quasi-Newton Method for Nonsmooth Regularized Optimization

TL;DR

This paper presents iR2N, an inexact modified quasi-Newton method for nonsmooth, possibly nonconvex optimization problems, which allows inexact evaluations and proximal computations to improve efficiency while ensuring convergence.

Contribution

The introduction of iR2N, a flexible inexact proximal quasi-Newton method that handles nonconvex nonsmooth optimization with convergence guarantees and reduced computational costs.

Findings

Effective in handling nonconvex nonsmooth problems.

Significant computational savings with controlled inexactness.

Convergence to first-order stationary points with complexity $O(\epsilon^{-2})$.

Abstract

We introduce iR2N, a modified proximal quasi-Newton method for minimizing the sum of a smooth function $f$ and a lower semi-continuous prox-bounded function $h$, allowing inexact evaluations of $f$, its gradient, and the associated proximal operators. Both $f$ and $h$ may be nonconvex. iR2N is particularly suited to settings where proximal operators are computed via iterative procedures that can be stopped early, or where the accuracy of $f$ and $\nabla f$ can be controlled, leading to significant computational savings. At each iteration, the method approximately minimizes the sum of a quadratic model of $f$, a model of $h$, and an adaptive quadratic regularization term ensuring global convergence. Under standard accuracy assumptions, we prove global convergence in the sense that a first-order stationarity measure converges to zero, with worst-case evaluation complexity $O(\epsilon^{-2})$. Numerical experiments with $\ell_p$ norms, $\ell_p$ total variation, and the indicator of the nonconvex pseudo $p$-norm ball illustrate the effectiveness and flexibility of the approach, and show how controlled inexactness can substantially reduce computational effort.