SPICE: Scaling-Aware Prediction Correction Methods with a Free Convergence Rate for Nonlinear Convex Optimization

TL;DR

The paper introduces SPICE, a scaling-aware prediction correction method that significantly improves convergence rates for nonlinear convex optimization by adaptively adjusting scaling factors, validated through theoretical analysis and numerical experiments.

Contribution

It proposes a novel scaling technique for prediction correction methods, achieving free and enhanced convergence rates in nonlinear convex optimization problems.

Findings

Achieves convergence rates of O(1/(t+1)), O(1/[e^{t}(t+1)]), and O(1/(t+1)^{t+1}) with different scaling strategies.

Theoretical analysis shows scaling adjustments greatly improve convergence.

Numerical experiments confirm the effectiveness of the proposed SPICE method.

Abstract

Recently, the prediction-correction method has been developed to solve nonlinear convex optimization problems. However, its convergence rate is often poor since large regularization parameters are set to ensure convergence conditions. In this paper, the scaling-aware prediction correction (\textsf{Spice}) method is proposed to achieve a free convergence rate. This method adopts a novel scaling technique that adjusts the weight of the objective and constraint functions. The theoretical analysis demonstrates that increasing the scaling factor for the objective function or decreasing the scaling factor for constraint functions significantly enhances the convergence rate of the prediction correction method. In addition, the \textsf{Spice} method is further extended to solve separable variable nonlinear convex optimization. By employing different scaling factors as functions of the iterations, the \textsf{Spice} method achieves convergence rates of $\mathcal{O}(1/(t+1))$, $\mathcal{O}(1/[e^{t}(t+1)])$, and $\mathcal{O}(1/(t+1)^{t+1})$. Numerical experiments further validate the theoretical findings, demonstrating the effectiveness of the \textsf{Spice} method in practice.