A Nesterov type algorithm with double Tikhonov regularization: fast convergence of the function values and strong convergence to the minimal norm solution

TL;DR

This paper introduces a Nesterov-type algorithm with double Tikhonov regularization that guarantees strong convergence to the minimal norm solution while maintaining optimal convergence rates for function values.

Contribution

The paper demonstrates that the proposed algorithm achieves strong convergence to the minimal norm solution without sacrificing Nesterov's optimal convergence rate.

Findings

Sequences converge strongly to the minimal norm element.

Optimal convergence rate of order O(n^{-2}) for function values is preserved.

Fast convergence of the discrete velocity and gradient estimates are established.

Abstract

We investigate the strong convergence properties of a Nesterov type algorithm with two Tikhonov regularization terms in connection to the minimization problem of a smooth convex function $f.$ We show that the generated sequences converge strongly to the minimal norm element from $\argmin f$. We also show that from a practical point of view the Tikhonov regularization does not affect Nesterov's optimal convergence rate of order $\mathcal{O}(n^{-2})$ for the potential energies $f(x_n)-\min f$ and $f(y_n)-\min f$, where $(x_n),\,(y_n)$ are the sequences generated by our algorithm. Further, we obtain fast convergence to zero of the discrete velocity, but also some estimates concerning the value of the gradient of the objective function in the generated sequences.