An accelerated gradient method with adaptive restart for convex multiobjective optimization problems

TL;DR

This paper introduces an accelerated gradient method with adaptive restart for convex multiobjective optimization, derived from a continuous-time approach, achieving improved convergence rates and practical efficiency.

Contribution

It develops a novel adaptive multiobjective gradient flow and a restart technique, extending existing methods with rigorous theoretical analysis and practical improvements.

Findings

Achieves fast sublinear and linear convergence rates for convex and strongly convex problems.

Provides a new continuous-time ODE model for multiobjective accelerated methods.

Demonstrates improved practical performance through numerical experiments.

Abstract

In this work, based on the continuous time approach, we propose an accelerated gradient method with adaptive residual restart for convex multiobjective optimization problems. For the first, we derive rigorously the continuous limit of the multiobjective accelerated proximal gradient method by Tanabe et al. [Comput. Optim. Appl., 2023]. It is a second-order ordinary differential equation (ODE) that involves a special projection operator and can be viewed as an extension of the ODE by Su et al. [J. Mach. Learn. Res., 2016] for Nesterov acceleration. Then, we introduce a novel accelerated multiobjective gradient (AMG) flow with tailored time scaling that adapts automatically to the convex case and the strongly convex case, and the exponential decay rate of a merit function along with the solution trajectory of AMG flow is established via the Lyapunov analysis. After that, we consider an implicit-explicit time discretization and obtain an accelerated multiobjective gradient method with a convex quadratic programming subproblem. The fast sublinear rate and linear rate are proved respectively for convex and strongly convex problems. In addition, we present an efficient residual based adaptive restart technique to overcome the oscillation issue and improve the convergence significantly. Numerical results are provided to validate the practical performance of the proposed method.