Gradient-Type Method for Optimization Problems with Polyak-Lojasiewicz Condition: Relative Inexactness in Gradient and Adaptive Parameters Setting

TL;DR

This paper introduces an adaptive gradient method for optimization problems satisfying the Polyak-Lojasiewicz condition, accounting for inexact gradients and adaptively tuning parameters, with theoretical guarantees and experimental validation.

Contribution

It proposes a novel adaptive gradient algorithm that handles relative inexactness in gradients and adaptively adjusts parameters for problems with Polyak-Lojasiewicz condition.

Findings

The algorithm achieves convergence guarantees under inexact gradient conditions.

Experimental results demonstrate the effectiveness of the adaptive approach.

Theoretical analysis confirms the quality of the obtained solutions.

Abstract

We consider minimization problems with the well-known Polya-Lojasievich condition and Lipshitz-continuous gradient. Such problem occurs in different places in machine learning and related fields. Furthermore, we assume that a gradient is available with some relative inexactness. We propose some adaptive gradient-type algorithm, where the adaptivity took place with respect to the smoothness parameter and the level of the gradient inexactness. The theoretical estimate of the the quality of the output point is obtained and backed up by experimental results.