Global Complexity Analysis of Inexact Successive Quadratic Approximation methods for Regularized Optimization under Mild Assumptions

TL;DR

This paper introduces a new framework for inexact successive quadratic approximation methods, providing improved global complexity results under milder assumptions for regularized optimization problems.

Contribution

It presents an algorithmic framework with four line search types and analyzes its global complexity, extending convergence guarantees under weaker conditions.

Findings

Linear convergence under quadratic growth and step size bounds.

Achieves o(1/k) complexity without quadratic growth condition.

Step size bounds linked to local gradient-Lipschitz continuity.

Abstract

Successive quadratic approximations (SQA) are numerically efficient for minimizing the sum of a smooth function and a convex function. The iteration complexity of inexact SQA methods has been analyzed recently. In this paper, we present an algorithmic framework of inexact SQA methods with four types of line searches, and analyze its global complexity under milder assumptions. First, we show its well-definedness and some decreasing properties. Second, under the quadratic growth condition and a uniform positive lower bound condition on stepsizes, we show that the function value sequence and the iterate sequence are linearly convergent. Moreover, we obtain a o(1/k) complexity without the quadratic growth condition, improving existing O(1/k) complexity results. At last, we show that a local gradient-Lipschitz-continuity condition could guarantee a uniform positive lower bound for the stepsizes.