A Gradient-Free Distributed Optimization Method for Convex Sum of Non-Convex Cost Functions

TL;DR

This paper introduces a gradient-free distributed optimization algorithm for convex sums of non-convex functions, leveraging Gaussian smoothing to handle non-smoothness and unknown gradients, with proven convergence and practical demonstrations.

Contribution

A novel gradient-free method using Gaussian smoothing for distributed optimization of convex sums of non-convex functions, with convergence guarantees.

Findings

Iterates converge to the optimal solution with probability 1 and in mean.

An upper bound on the optimality gap is established.

Numerical and privacy applications demonstrate effectiveness.

Abstract

This paper presents a special type of distributed optimization problems, where the summation of agents' local cost functions (i.e., global cost function) is convex, but each individual can be non-convex. Unlike most distributed optimization algorithms by taking the advantages of gradient, the considered problem is allowed to be non-smooth, and the gradient information is unknown to the agents. To solve the problem, a Gaussian-smoothing technique is introduced and a gradient-free method is proposed. We prove that each agent's iterate approximately converges to the optimal solution both with probability 1 and in mean, and provide an upper bound on the optimality gap, characterized by the difference between the functional value of the iterate and the optimal value. The performance of the proposed algorithm is demonstrated by a numerical example and an application in privacy enhancement.