The Cyclic Block Conditional Gradient Method for Convex Optimization Problems

TL;DR

This paper introduces a cyclic block conditional gradient method for convex optimization problems involving a smooth and a separable non-smooth term, providing convergence analysis and demonstrating its effectiveness through numerical experiments.

Contribution

It develops a cyclic block variant of the generalized conditional gradient method with proven convergence rates and empirical advantages over existing algorithms.

Findings

Achieves a global sublinear convergence rate.

Outperforms classical conditional gradient and random block methods in experiments.

Effective for convex problems with separable non-smooth terms.

Abstract

In this paper we study the convex problem of optimizing the sum of a smooth function and a compactly supported non-smooth term with a specific separable form. We analyze the block version of the generalized conditional gradient method when the blocks are chosen in a cyclic order. A global sublinear rate of convergence is established for two different stepsize strategies commonly used in this class of methods. Numerical comparisons of the proposed method to both the classical conditional gradient algorithm and its random block version demonstrate the effectiveness of the cyclic block update rule.