$\rho$-regularization subproblems: Strong duality and an eigensolver-based algorithm

TL;DR

This paper introduces a general regularized subproblem framework encompassing trust-region and p-regularization problems, establishes strong duality, and proposes an eigensolver-based algorithm with promising numerical results.

Contribution

It develops a unified theory for $ ho$-regularized subproblems, including strong duality and optimality conditions, and introduces an eigensolver-based method for solving the dual problem.

Findings

Strong duality holds for $ ho$RS under general conditions.

The eigensolver algorithm efficiently solves the dual problem.

Numerical experiments demonstrate the algorithm's effectiveness.

Abstract

Trust-region (TR) type method, based on a quadratic model such as the trust-region subproblem (TRS) and $ p $-regularization subproblem ($p$RS), is arguably one of the most successful methods for unconstrained minimization. In this paper, we study a general regularized subproblem (named $ \rho $RS), which covers TRS and $p$RS as special cases. We derive a strong duality theorem for $ \rho $RS, and also its necessary and sufficient optimality condition under general assumptions on the regularization term. We then define the Rendl-Wolkowicz (RW) dual problem of $ \rho $RS, which is a maximization problem whose objective function is concave, and differentiable except possibly at two points. It is worth pointing out that our definition is based on an alternative derivation of the RW-dual problem for TRS. Then we propose an eigensolver-based algorithm for solving the RW-dual problem of $ \rho $RS. The algorithm is carried out by finding the smallest eigenvalue and its unit eigenvector of a certain matrix in each iteration. Finally, we present numerical results on randomly generated $p$RS's, and on a new class of regularized problem that combines TRS and $p$RS, to illustrate our algorithm.