A Regularized Limited Memory Subspace Minimization Conjugate Gradient Method for Unconstrained Optimization

TL;DR

This paper introduces a novel regularized limited memory subspace minimization conjugate gradient method for unconstrained optimization, combining multiple techniques to improve convergence and efficiency, with demonstrated superior performance over existing algorithms.

Contribution

It proposes a new algorithm integrating regularization, subspace minimization, and acceleration strategies, with proven global convergence and enhanced numerical performance.

Findings

Significant improvement over ASMCG_PR.

Outperforms CG_DESCENT and CGOPT on CUTEr benchmark.

Effective in reducing iteration count and computational time.

Abstract

In this paper, based on the limited memory techniques and subspace minimization conjugate gradient (SMCG) methods, a regularized limited memory subspace minimization conjugate gradient method is proposed, which contains two types of iterations. In SMCG iteration, we obtain the search direction by minimizing the approximate quadratic model or approximate regularization model. In RQN iteration, combined with regularization technique and BFGS method, a modified regularized quasi-Newton method is used in the subspace to improve the orthogonality. Moreover, some simple acceleration criteria and an improved tactic for selecting the initial stepsize to enhance the efficiency of the algorithm are designed. Additionally, an generalized nonmonotone line search is utilized and the global convergence of our proposed algorithm is established under mild conditions. Finally, numerical results show that, the proposed algorithm has a significant improvement over ASMCG_PR and is superior to the particularly well-known limited memory conjugate gradient software packages CG_DESCENT (6.8) and CGOPT(2.0) for the CUTEr library.