On the Robustness of the Successive Projection Algorithm

TL;DR

This paper analyzes the noise robustness of the Successive Projection Algorithm (SPA) and its variants, providing tight and improved error bounds, and proposing a new robust variant with enhanced stability for learning convex hull vertices.

Contribution

The paper offers the first tight error bounds for SPA when r ≥ 3, improves existing bounds in special cases, and introduces a new robust SPA variant that enhances stability through data shifting and lifting.

Findings

Tight error bounds established for SPA when r ≥ 3.

Improved error bounds for SPA in specific cases related to vertex conditioning.

A new robust SPA variant that minimizes problem conditioning through data transformation.

Abstract

The successive projection algorithm (SPA) is a workhorse algorithm to learn the $r$ vertices of the convex hull of a set of $(r-1)$-dimensional data points, a.k.a. a latent simplex, which has numerous applications in data science. In this paper, we revisit the robustness to noise of SPA and several of its variants. In particular, when $r \geq 3$, we prove the tightness of the existing error bounds for SPA and for two more robust preconditioned variants of SPA. We also provide significantly improved error bounds for SPA, by a factor proportional to the conditioning of the $r$ vertices, in two special cases: for the first extracted vertex, and when $r \leq 2$. We then provide further improvements for the error bounds of a translated version of SPA proposed by Arora et al. (''A practical algorithm for topic modeling with provable guarantees'', ICML, 2013) in two special cases: for the first two extracted vertices, and when $r \leq 3$. Finally, we propose a new more robust variant of SPA that first shifts and lifts the data points in order to minimize the conditioning of the problem. We illustrate our results on synthetic data.