Extended-Krylov-subspace methods for trust-region and norm-regularization subproblems

TL;DR

This paper introduces TREK/NREK, an efficient extended-Krylov subspace method for solving trust-region and norm-regularization subproblems, reducing computational cost by leveraging low-dimensional solution spaces.

Contribution

The paper presents a novel extended-Krylov subspace approach that efficiently solves subproblems with a single matrix factorization, improving over existing methods.

Findings

Effective low-dimensional solution representation.

Reduced computational cost with a single matrix factorization.

Numerical results demonstrate method's efficiency.

Abstract

We consider an effective new method for solving trust-region and norm-regularization problems that arise as subproblems in many optimization applications. We show that the solutions to such subproblems effectively lie in a very-low-dimensional subspace as a function of their controlling parameters (trust-region radius or regularization weight). Based on this, we build a basis spanning these solutions using an efficient extended-Krylov-subspace iteration that involves a single matrix factorization. The problems within the subspace using such a basis may be solved at very low cost using effective high-order root-finding methods. This then provides an alternative to common methods using multiple factorizations or standard Krylov subspaces. We provide numerical results to illustrate the effectiveness of our TREK/NREK approach.