Projection free methods on product domains

TL;DR

This paper develops a new convergence theory for projection-free block-coordinate methods on product domains, extending to non-convex objectives and leveraging sparsity for efficient solutions.

Contribution

It introduces a comprehensive convergence framework for non-convex projection-free methods on product domains, bridging a theoretical gap and enabling exploitation of solution sparsity.

Findings

Preliminary experiments support the effectiveness of the proposed methods.

The approach shows promise for obtaining global solutions in non-convex settings.

New active set identification results improve solution sparsity exploitation.

Abstract

Projection-free block-coordinate methods avoid high computational cost per iteration and at the same time exploit the particular problem structure of product domains. Frank-Wolfe-like approaches rank among the most popular ones of this type. However, as observed in the literature, there was a gap between the classical Frank-Wolfe theory and the block-coordinate case. Moreover, most of previous research concentrated on convex objectives. This study now deals also with the non-convex case and reduces above-mentioned theory gap, in combining a new, fully developed convergence theory with novel active set identification results which ensure that inherent sparsity of solutions can be exploited in an efficient way. Preliminary numerical experiments seem to justify our approach and also show promising results for obtaining global solutions in the non-convex case.