Abs-Smooth Frank-Wolfe Method: Primal-Dual Analysis, Heavy Ball Momentum, and Inexact Oracles

TL;DR

This paper introduces a unified Abs-Smooth Frank-Wolfe framework for convex optimization that handles non-smooth functions, incorporates momentum, and is robust to inexact oracles, broadening applicability.

Contribution

It extends Frank-Wolfe methods to abs-smooth functions, includes momentum, and handles inexact oracles, relaxing convexity assumptions for wider non-smooth problem classes.

Findings

Established convergence guarantees without classical smoothness assumptions.

Incorporated heavy ball momentum while maintaining convergence.

Demonstrated robustness to approximate inner solutions.

Abstract

We study projection-free optimization for convex objectives that satisfy abs-smoothness, a structural property that captures many non-smooth yet piecewise smooth functions arising, e.g., in modern machine learning models. We develop a unified framework for Abs-Smooth Frank-Wolfe methods, establishing a clean primal-dual analysis that guarantees convergence without requiring classical smoothness assumptions. Our framework extends the available results in two important directions. First, we introduce a heavy ball momentum variant and show that momentum can be incorporated naturally under abs-smoothness while preserving convergence guarantees. Second, we analyze inexact minimization oracles, demonstrating robustness to approximate inner solutions. Moreover, we relax the full convexity assumption and study the case where convexity holds only for the piecewise linear approximations of the objective, further broadening the applicability of conditional gradient methods to a wider class of non-smooth problems.