A smoothed proximal trust-region algorithm for nonconvex optimization problems with $L^p$-regularization, $p\in (0,1)$

TL;DR

This paper introduces a novel smoothed proximal trust-region algorithm for nonconvex optimization with $L^p$-regularization ($p ext{ in } (0,1)$), combining smoothing, convex upper bounds, and trust-region methods to ensure convergence.

Contribution

It develops a new algorithm that effectively handles nonconvex, nonsmooth $L^p$-regularization problems using smoothing and convex upper bounds within a trust-region framework.

Findings

Proves convergence properties of the proposed algorithm.

Demonstrates the algorithm's effectiveness through numerical examples.

Discusses approximate subproblem solvers for trust-region subproblems.

Abstract

We investigate a trust-region algorithm to solve a nonconvex optimization problem with $L^p$-regularization for $p\in(0,1)$. The algorithm relies on descent properties of a so-called generalized Cauchy point that can be obtained efficiently by a line search along a suitable proximal path. To handle the nonconvexity and nonsmoothness of the $L^p$-pseudonorm, we replace it by a smooth approximation and construct a convex upper bound of that approximation. This enables us to use results of a trust-region method for composite problems with a convex nonsmooth term. We prove convergence properties of the resulting smoothed proximal trust-region algorithm and investigate its performance in some numerical examples. Furthermore, approximate subproblem solvers for the arising trust-region subproblems are considered.