A convergence analysis of the method of codifferential descent

TL;DR

This paper provides a comprehensive convergence analysis of the method of codifferential descent (MCD), introduces a generalized and regularized version, and discusses its robustness and convergence rates for nonsmooth nonconvex optimization.

Contribution

It introduces a generalized, more application-friendly MCD, proves its global convergence in infinite dimensions, and analyzes a novel quadratic regularization method.

Findings

The generalized MCD converges globally in infinite-dimensional spaces.

The regularized MCD is the first method for minimizing codifferentiable functions over convex sets.

The MCD can achieve linear and quadratic convergence rates under certain conditions.

Abstract

This paper is devoted to a detailed convergence analysis of the method of codifferential descent (MCD) developed by professor V.F. Demyanov for solving a large class of nonsmooth nonconvex optimization problems. We propose a generalization of the MCD that is more suitable for applications than the original method, and that utilizes only a part of a codifferential on every iteration, which allows one to reduce the overall complexity of the method. With the use of some general results on uniformly codifferentiable functions obtained in this paper, we prove the global convergence of the generalized MCD in the infinite dimensional case. Also, we propose and analyse a quadratic regularization of the MCD, which is the first general method for minimizing a codifferentiable function over a convex set. Apart from convergence analysis, we also discuss the robustness of the MCD with respect to computational errors, possible step size rules, and a choice of parameters of the algorithm. In the end of the paper we estimate the rate of convergence of the MCD for a class of nonsmooth nonconvex functions that arises, in particular, in cluster analysis. We prove that under some general assumptions the method converges with linear rate, and it convergence quadratically, provided a certain first order sufficient optimality condition holds true.