CCCP is Frank-Wolfe in disguise

TL;DR

This paper reveals that the convex-concave procedure (CCCP) is essentially a form of the Frank-Wolfe method, enabling the transfer of convergence theories and insights between these algorithms.

Contribution

It establishes a fundamental connection between CCCP and Frank-Wolfe, providing new theoretical understanding and potential for cross-application of recent convergence results.

Findings

CCCP is a special case of Frank-Wolfe.

Non-asymptotic convergence theory of FW applies to CCCP.

This connection offers pedagogical and theoretical insights.

Abstract

This paper uncovers a simple but rather surprising connection: it shows that the well-known convex-concave procedure (CCCP) and its generalization to constrained problems are both special cases of the Frank-Wolfe (FW) method. This connection not only provides insight of deep (in our opinion) pedagogical value, but also transfers the recently discovered convergence theory of nonconvex Frank-Wolfe methods immediately to CCCP, closing a long-standing gap in its non-asymptotic convergence theory. We hope the viewpoint uncovered by this paper spurs the transfer of other advances made for FW to both CCCP and its generalizations.