Symplectic Discretization Approach for Developing New Proximal Point Algorithm

TL;DR

This paper introduces a new accelerated proximal point algorithm derived from a symplectic discretization of an ODE, which improves convergence and reduces oscillations in high-dimensional optimization tasks.

Contribution

It develops the Symplectic Proximal Point Algorithm using symplectic Euler discretization, achieving better convergence rates and numerical stability compared to existing methods.

Findings

Achieves an o(1/k^2) convergence rate.

Reduces oscillatory behavior in numerical experiments.

Demonstrates weak convergence to the solution set.

Abstract

The rapid advancements in high-dimensional statistics and machine learning have increased the use of first-order methods. Many of these methods can be regarded as instances of the proximal point algorithm. Given the importance of the proximal point algorithm, there has been growing interest in developing its accelerated variants. However, some existing accelerated proximal point algorithms exhibit oscillatory behavior, which can impede their numerical convergence rate. In this paper, we first introduce an ODE system and demonstrate its \( o(1/t^2) \) convergence rate and weak convergence property. Next, we apply the Symplectic Euler Method to discretize the ODE and obtain a new accelerated proximal point algorithm, which we call the Symplectic Proximal Point Algorithm. The reason for using the Symplectic Euler Method is its ability to preserve the geometric structure of the ODEs. Theoretically, we demonstrate that the Symplectic Proximal Point Algorithm achieves an \( o(1/k^2) \) convergence rate and that the sequences generated by our method converge weakly to the solution set. Practically, our numerical experiments illustrate that the Symplectic Proximal Point Algorithm significantly reduces oscillatory behavior, leading to improved long-time behavior and faster numerical convergence rate.