Accelerated Distance-adaptive Methods for H\"{o}lder Smooth and Convex Optimization

TL;DR

This paper develops new parameter-free, accelerated first-order methods for H"{o}lder smooth convex optimization that adaptively achieve optimal convergence without prior knowledge of smoothness parameters or explicit tuning.

Contribution

It introduces an accelerated distance-adaptive method that is parameter-free, removing the need for prior smoothness knowledge and explicit parameter tuning in convex optimization.

Findings

Achieves optimal convergence rates for H"{o}lder smooth problems.

Removes the need for line-search in stochastic convex optimization.

Demonstrates effectiveness on nonsmooth convex problems.

Abstract

This paper introduces new parameter-free first-order methods for convex optimization problems in which the objective function exhibits H\"{o}lder smoothness. Inspired by the recently proposed distance-over-gradient (DOG) technique, we propose an accelerated distance-adaptive method which achieves optimal anytime convergence rates for H\"{o}lder smooth problems without requiring prior knowledge of smoothness parameters or explicit parameter tuning. Importantly, our parameter-free approach removes the necessity of specifying target accuracy in advance, addressing a limitation found in the universal fast gradient methods (Nesterov, Yu. \textit{Mathematical Programming}, 2015). For convex stochastic optimization, we further present a parameter-free accelerated method that eliminates the need for line-search procedures. Preliminary experimental results highlight the effectiveness of our approach on convex nonsmooth problems and its advantages over existing parameter-free or accelerated methods.