A unified way of analyzing some greedy algorithms

TL;DR

This paper introduces a unified analytical framework for a broad class of greedy algorithms in Banach spaces, including new variants, and studies their convergence, stability, and efficiency under computational inaccuracies.

Contribution

It defines the Weak Biorthogonal Greedy Algorithms class, unifies analysis of known algorithms, and proposes a new algorithm with proven convergence properties.

Findings

The class includes Weak Chebyshev and Weak Greedy Algorithms.

Convergence and stability depend on computational inaccuracies.

New Rescaled Weak Relaxed Greedy Algorithm demonstrates effective convergence.

Abstract

In this paper we propose a unified way of analyzing a certain kind of greedy-type algorithms in Banach spaces. We define a class of the Weak Biorthogonal Greedy Algorithms that contains a wide range of greedy algorithms. In particular, we show that the following well-known algorithms --- the Weak Chebyshev Greedy Algorithm and the Weak Greedy Algorithm with Free Relaxation --- belong to this class. We investigate the properties of convergence, rate of convergence, and numerical stability of the Weak Biorthogonal Greedy Algorithms. Numerical stability is understood in the sense that the steps of the algorithm are allowed to be performed with controlled computational inaccuracies. We carry out a thorough analysis of the connection between the magnitude of those inaccuracies and the convergence properties of the algorithm. To emphasize the advantage of the proposed approach, we introduce here a new greedy algorithm --- the Rescaled Weak Relaxed Greedy Algorithm --- from the above class, and derive the convergence results without analyzing the algorithm explicitly. Additionally, we explain how the proposed approach can be extended to some other types of greedy algorithms.