Unconstrained optimization using the directional proximal point method

TL;DR

This paper introduces a directional proximal point method (DPPM) for unconstrained optimization that efficiently finds critical points of smooth functions, especially suitable for large-scale problems, with proven convergence properties.

Contribution

The paper proposes a novel DPPM that determines descent directions via a 2D quadratic problem, enabling scalable optimization for high-dimensional functions.

Findings

DPPM converges to critical points of the function.

The entire DPPM sequence can converge to a single critical point under certain conditions.

For strongly convex quadratic functions, convergence can be R-superlinear regardless of dimension.

Abstract

This paper presents a directional proximal point method (DPPM) to derive the minimum of any C1-smooth function f. The proposed method requires a function persistent a local convex segment along the descent direction at any non-critical point (referred to a DLC direction at the point). The proposed DPPM can determine a DLC direction by solving a two-dimensional quadratic optimization problem, regardless of the dimensionally of the function variables. Along that direction, the DPPM then updates by solving a one-dimensional optimization problem. This gives the DPPM advantage over competitive methods when dealing with large-scale problems, involving a large number of variables. We show that the DPPM converges to critical points of f. We also provide conditions under which the entire DPPM sequence converges to a single critical point. For strongly convex quadratic functions, we demonstrate that the rate at which the error sequence converges to zero can be R-superlinear, regardless of the dimension of variables.