New Proximal Newton-Type Methods for Convex Optimization

TL;DR

This paper introduces novel proximal Newton-type methods for convex optimization that balance computational efficiency with fast convergence, suitable for applications like MPC.

Contribution

The paper presents new proximal Newton methods that avoid Hessian evaluations at each step while maintaining superlinear convergence, including practical variants with quasi-Newton and inexact schemes.

Findings

Guaranteed global convergence.

Achieved superlinear convergence near solutions.

Demonstrated effectiveness on real-world datasets.

Abstract

In this paper, we propose new proximal Newton-type methods for convex optimization problems in composite form. The applications include model predictive control (MPC) and embedded MPC. Our new methods are computationally attractive since they do not require evaluating the Hessian at each iteration while keeping fast convergence rate. More specifically, we prove the global convergence is guaranteed and the superlinear convergence is achieved in the vicinity of an optimal solution. We also develop several practical variants by incorporating quasi-Newton and inexact subproblem solving schemes and provide theoretical guarantee for them under certain conditions. Experimental results on real-world datasets demonstrate the effectiveness and efficiency of new methods.