Smoothing Iterative Consensus-based Optimization Algorithm for Nonsmooth Nonconvex Optimization Problems with Global Optimality

TL;DR

This paper introduces a novel smoothing iterative consensus-based optimization (SICBO) algorithm that guarantees convergence to the global minimizer for nonsmooth, nonconvex, and possibly non-Lipschitz functions, with theoretical and numerical validation.

Contribution

The paper presents the first theoretical guarantee of global optimality for a consensus-based algorithm applied to nonsmooth, nonconvex, and non-Lipschitz optimization problems.

Findings

Exponential convergence to a stochastic consensus point almost surely.

Theoretical bounds on the error between the consensus objective and the global minimum.

Numerical experiments demonstrating the algorithm's effectiveness in complex optimization scenarios.

Abstract

In this paper, we focus on finding the global minimizer of a general unconstrained nonsmooth nonconvex optimization problem. Taking advantage of the smoothing method and the consensus-based optimization (CBO) method, we propose a novel smoothing iterative consensus-based optimization (SICBO) algorithm. First, we prove that the solution process of the proposed algorithm here exponentially converges to a common stochastic consensus point almost surely. Second, we establish a detailed theoretical analysis to ensure the small enough error between the objective function value at the consensus point and the optimal function value, to the best of our knowledge, which provides the first theoretical guarantee to the global optimality of the proposed algorithm for nonconvex optimization problems. Moreover, unlike the previously introduced CBO methods, the theoretical results are valid for the cases that the objective function is nonsmooth, nonconvex and perhaps non-Lipschitz continuous. Finally, several numerical examples are performed to illustrate the effectiveness of our proposed algorithm for solving the global minimizer of the nonsmooth and nonconvex optimization problems.