Distributed Derivative-free Learning Method for Stochastic Optimization over a Network with Sparse Activity

TL;DR

This paper introduces a distributed derivative-free learning algorithm for stochastic optimization in networks with sporadically active nodes, addressing challenges of limited information and stochastic disturbances, and proves its convergence.

Contribution

It proposes a novel distributed derivative-free method for stochastic optimization over networks with sparse activity, handling non-additive noise and limited utility information.

Findings

Algorithm converges almost surely to the global optimum.

Convergence rate is established under strong concavity.

Numerical experiments validate theoretical results.

Abstract

This paper addresses a distributed optimization problem in a communication network where nodes are active sporadically. Each active node applies some learning method to control its action to maximize the global utility function, which is defined as the sum of the local utility functions of active nodes. We deal with stochastic optimization problem with the setting that utility functions are disturbed by some non-additive stochastic process. We consider a more challenging situation where the learning method has to be performed only based on a scalar approximation of the utility function, rather than its closed-form expression, so that the typical gradient descent method cannot be applied. This setting is quite realistic when the network is affected by some stochastic and time-varying process, and that each node cannot have the full knowledge of the network states. We propose a distributed optimization algorithm and prove its almost surely convergence to the optimum. Convergence rate is also derived with an additional assumption that the objective function is strongly concave. Numerical results are also presented to justify our claim.