A Parameter-Free Conditional Gradient Method for Composite Minimization under H\"older Condition

TL;DR

This paper introduces a parameter-free conditional gradient method for composite optimization problems, eliminating the need for prior knowledge of problem parameters and achieving competitive convergence rates.

Contribution

The paper proposes a novel parameter-free conditional gradient algorithm that adaptively determines step sizes without prior parameter knowledge, matching the convergence rate of parameter-dependent methods.

Findings

Achieves the same convergence rate as parameter-dependent methods

Demonstrates superior performance in preliminary experiments

Eliminates the need for prior parameter knowledge

Abstract

In this paper we consider a composite optimization problem that minimizes the sum of a weakly smooth function and a convex function with either a bounded domain or a uniformly convex structure. In particular, we first present a parameter-dependent conditional gradient method for this problem, whose step sizes require prior knowledge of the parameters associated with the H\"older continuity of the gradient of the weakly smooth function, and establish its rate of convergence. Given that these parameters could be unknown or known but possibly conservative, such a method may suffer from implementation issue or slow convergence. We therefore propose a parameter-free conditional gradient method whose step size is determined by using a constructive local quadratic upper approximation and an adaptive line search scheme, without using any problem parameter. We show that this method achieves the same rate of convergence as the parameter-dependent conditional gradient method. Preliminary experiments are also conducted and illustrate the superior performance of the parameter-free conditional gradient method over the methods with some other step size rules.