Accelerated Inertial Gradient Algorithms with Vanishing Tikhonov Regularization

TL;DR

This paper introduces an accelerated inertial gradient algorithm with vanishing Tikhonov regularization, achieving fast convergence and strong convergence to minimum-norm solutions in convex optimization, supported by theoretical analysis and numerical experiments.

Contribution

It develops a new explicit Tikhonov-regularized inertial gradient method with proven accelerated convergence and strong convergence properties under specific regularization decay schedules.

Findings

Achieves accelerated convergence of objective values.

Ensures strong convergence to minimum-norm solutions for certain decay rates.

Demonstrates practical effectiveness through numerical experiments.

Abstract

In this paper, we study an explicit Tikhonov-regularized inertial gradient algorithm for smooth convex minimization with Lipschitz continuous gradient. The method is derived via an explicit time discretization of a damped inertial system with vanishing Tikhonov regularization. Under appropriate control of the decay rate of the Tikhonov term, we establish accelerated convergence of the objective values to the minimum together with strong convergence of the iterates to the minimum-norm minimizer. In particular, for polynomial schedules $\varepsilon_k = k^{-p}$ with $0<p<2$, we prove strong convergence to the minimum-norm solution while preserving fast objective decay. In the critical case $p=2$, we still obtain fast rates for the objective values, while our analysis does not guarantee strong convergence to the minimum-norm minimizer. Furthermore, we provide a thorough theoretical analysis for several choices of Tikhonov schedules. Numerical experiments on synthetic, benchmark, and real datasets illustrate the practical performance of the proposed algorithm.