A Modified Nonlinear Conjugate Gradient Algorithm for Functions with Non-Lipschitz Gradient

TL;DR

This paper introduces a modified nonlinear conjugate gradient method tailored for functions with non-Lipschitz continuous gradients, ensuring convergence and broadening applicability while maintaining strong numerical performance.

Contribution

A new conjugate coefficient formula and an interpolation approach are proposed, enabling effective optimization for non-Lipschitz functions with guaranteed convergence.

Findings

Guarantees strong convergence for non-Lipschitz functions

Automatically achieves Wolfe conditions in each step

Broadens applicability of NCG methods

Abstract

In this paper, we propose a modified nonlinear conjugate gradient (NCG) method for functions with a non-Lipschitz continuous gradient. First, we present a new formula for the conjugate coefficient \beta_k in NCG, conducting a search direction that provides an adequate function decrease. We can derive that our NCG algorithm guarantees strongly convergent for continuous differential functions without Lipschitz continuous gradient. Second, we present a simple interpolation approach that could automatically achieve shrinkage, generating a step length satisfying the standard Wolfe conditions in each step. Our framework considerably broadens the applicability of NCG and preserves the superior numerical performance of the PRP-type methods.