Inexact Limited Memory Bundle Method

TL;DR

This paper introduces a new inexact limited memory bundle method designed for large-scale nonsmooth nonconvex optimization, capable of handling noisy function and subgradient data with proven convergence.

Contribution

It presents a novel optimization algorithm that tolerates inexact information and demonstrates its effectiveness through theoretical convergence and numerical experiments.

Findings

Method converges to an approximate stationary point despite noise.

Performs well with both exact and noisy data in large-scale problems.

Shows competitiveness and suitability for privacy-preserving machine learning applications.

Abstract

Large-scale nonsmooth optimization problems arise in many real-world applications, but obtaining exact function and subgradient values for these problems may be computationally expensive or even infeasible. In many practical settings, only inexact information is available due to measurement or modeling errors, privacy-preserving computations, or stochastic approximations, making inexact optimization methods particularly relevant. In this paper, we propose a novel inexact limited memory bundle method for large-scale nonsmooth nonconvex optimization. The method tolerates noise in both function values and subgradients. We prove the global convergence of the proposed method to an approximate stationary point. Numerical experiments with different levels of noise in function and/or subgradient values show that the method performs well with both exact and noisy data. In particular, the results demonstrate competitiveness in large-scale nonsmooth optimization and highlight the suitability of the method for applications where noise is unavoidable, such as differential privacy in machine learning.