Coderivative-Based Newton Methods with Wolfe Linesearch for Nonsmooth Optimization

TL;DR

This paper develops novel coderivative-based Newton methods with Wolfe linesearch for nonsmooth optimization, achieving global convergence and applying to machine learning problems like Lasso and SVM.

Contribution

It introduces new Newton-type algorithms for nonsmooth problems, combining coderivative techniques with Wolfe linesearch and extending to constrained convex composite optimization.

Findings

Algorithms demonstrate superlinear convergence.

Effective in solving Lasso and SVM problems.

Numerical results show high efficiency.

Abstract

This paper introduces and develops novel coderivative-based Newton methods with Wolfe linesearch conditions to solve various classes of problems in nonsmooth optimization. We first propose a generalized regularized Newton method with Wolfe linesearch (GRNM-W) for unconstrained $C^{1,1}$ minimization problems (which are second-order nonsmooth) and establish global as well as local superlinear convergence of their iterates. To deal with convex composite minimization problems (which are first-order nonsmooth and can be constrained), we combine the proposed GRNM-W with two algorithmic frameworks: the forward-backward envelope and the augmented Lagrangian method resulting in the two new algorithms called CNFB and CNAL, respectively. Finally, we present numerical results to solve Lasso and support vector machine problems appearing in, e.g., machine learning and statistics, which demonstrate the efficiency of the proposed algorithms.