Convergence Analysis of Greedy Algorithms with Adaptive Relaxation in Hilbert Spaces

TL;DR

This paper investigates the convergence properties of a generalized greedy algorithm with adaptive relaxation in Hilbert spaces, addressing open questions for certain parameter ranges and proposing an optimal step size variant.

Contribution

It provides a convergence analysis for the Power-Relaxed Greedy Algorithm with lpha>1 and introduces an adaptive step size method via exact line search.

Findings

Convergence rates are established for lpha>1.

Optimal step size via line search improves algorithm performance.

Addresses open problem in greedy algorithm behavior for lpha>1.

Abstract

The Power-Relaxed Greedy Algorithm (PRGA) was introduced as a generalization of the so called Relaxed Greedy Algorithm, introduced by DeVore and Temlyakov, by replacing the relaxation parameter $1/m$ with $1/m^\alpha$, with the aim of improving convergence rates. While the case $\alpha\le 1$ is well understood, the behavior of the algorithm for $\alpha>1$ remained an open problem. In this work, we answer this question and, moreover, we introduce a relaxed greedy algorithm with an optimal step size chosen by exact line search at each iteration.