A hybrid proximal generalized conditional gradient method and application to total variation parameter learning

TL;DR

This paper introduces a hybrid optimization method combining forward-backward and generalized conditional gradient techniques, with proven convergence rates, applied to total variation parameter learning to enhance nonsmooth convex optimization tasks.

Contribution

A novel hybrid proximal generalized conditional gradient method with convergence guarantees, tailored for convex problems involving smooth and nonsmooth functions, demonstrated on total variation parameter learning.

Findings

Convergence rate of o(k^{-1/3}) under mild conditions

Effective application to total variation parameter learning

Improved optimization performance in nonsmooth convex problems

Abstract

In this paper we present a new method for solving optimization problems involving the sum of two proper, convex, lower semicontinuous functions, one of which has Lipschitz continuous gradient. The proposed method has a hybrid nature that combines the usual forward-backward and the generalized conditional gradient method. We establish a convergence rate of $o(k^{-1/3})$ under mild assumptions with a specific step-size rule and show an application to a total variation parameter learning problem, which demonstrates its benefits in the context of nonsmooth convex optimization.