A multilevel proximal trust-region method for nonsmooth optimization with applications

TL;DR

This paper introduces a multilevel proximal trust-region method designed for nonsmooth optimization problems, effectively combining multilevel strategies with proximal methods to improve convergence and computational efficiency in large-scale applications.

Contribution

It unifies multilevel and proximal trust-region approaches, providing a novel algorithm with proven global convergence for nonsmooth, nonconvex optimization problems.

Findings

Reduces computational cost in step calculation.

Proven global convergence in finite-dimensional spaces.

Demonstrated efficiency in PDE-constrained optimization and machine learning.

Abstract

Many large-scale optimization problems arising in science and engineering are naturally defined at multiple levels of discretization or model fidelity. Multilevel methods exploit this hierarchy to accelerate convergence by combining coarse- and fine-level information, a strategy that has proven highly effective in the numerical solution of partial differential equations and related optimization problems. It turns out that many applications in PDE-constrained optimization and data science require minimizing the sum of smooth and nonsmooth functions. For example, training neural networks may require minimizing a mean squared error plus an $L^1$-regularization to induce sparsity in the weights. Correspondingly, we introduce a multilevel proximal trust-region method to minimize the sum of a nonconvex, smooth and a convex, nonsmooth function. Exploiting ideas from the multilevel literature allows us to reduce the cost of the step computation, which is a major bottleneck in single level procedures. Our work unifies theory behind the proximal trust-region methods and multilevel recursive strategies. We prove global convergence of our method in finite dimensional space and provide an efficient nonsmooth subproblem solver. We show the efficiency and robustness of our algorithm by means of numerical examples in PDE constrained optimization and machine-learning.