An Element-wise RSAV Algorithm for Unconstrained Optimization Problems

TL;DR

This paper introduces the E-RSAV algorithm for unconstrained optimization, ensuring energy dissipation, with proven linear convergence and an accelerated super-linear version, validated by extensive experiments.

Contribution

The paper proposes a novel element-wise RSAV algorithm with energy dissipation and convergence guarantees, including an accelerated and adaptive variant for improved performance.

Findings

Proves linear convergence in convex settings.

Develops an accelerated super-linear version for univariate problems.

Demonstrates robustness and fast convergence through numerical experiments.

Abstract

We present a novel optimization algorithm, element-wise relaxed scalar auxiliary variable (E-RSAV), that satisfies an unconditional energy dissipation law and exhibits improved alignment between the modified and the original energy. Our algorithm features rigorous proofs of linear convergence in the convex setting. Furthermore, we present a simple accelerated algorithm that improves the linear convergence rate to super-linear in the univariate case. We also propose an adaptive version of E-RSAV with Steffensen step size. We validate the robustness and fast convergence of our algorithm through ample numerical experiments.