A Fully Adaptive Frank-Wolfe Algorithm for Relatively Smooth Problems and Its Application to Centralized Distributed Optimization

TL;DR

This paper introduces a fully adaptive Frank-Wolfe algorithm for constrained optimization with relatively smooth objectives, which does not require prior parameter knowledge and guarantees convergence, with applications to distributed optimization.

Contribution

It proposes a novel adaptive Frank-Wolfe method that adjusts step sizes dynamically and applies to distributed optimization problems with relative smoothness and strong convexity.

Findings

Achieves linear convergence under relative strong convexity.

Demonstrates accelerated convergence in distributed optimization scenarios.

Provides theoretical guarantees for the adaptive step-size rule.

Abstract

We study the Frank-Wolfe algorithm for constrained optimization problems with relatively smooth objectives. Building upon our previous work, we propose a fully adaptive variant of the Frank-Wolfe method that dynamically adjusts the step size. Our method does not require prior knowledge of the function parameters and guarantees convergence using only local information. We establish a linear convergence rate under relative strong convexity and provide a detailed theoretical analysis of the proposed adaptive step-size rule. Furthermore, we demonstrate how relative smoothness and strong convexity naturally arise in the setting of centralized distributed optimization. Under a variance-type assumption on the gradients, we show that the global objective becomes relatively strongly convex with respect to the Bregman divergence generated by a local function. This structure allows us to apply our adaptive Frank-Wolfe algorithm, leading to provable acceleration due to an improved relative condition number.