Distributed Optimization Over Markovian Switching Random Network

TL;DR

This paper introduces a distributed sub-gradient algorithm for convex optimization over multi-agent systems with Markovian switching networks, ensuring convergence despite network uncertainties.

Contribution

It proposes a novel two time-scale step-size consensus algorithm that guarantees convergence over Markovian switching directed graphs.

Findings

Proves almost sure convergence under certain connectivity conditions.

Handles uncertainty due to Markovian network switching.

Validates results through simulations.

Abstract

In this paper, we investigate the distributed convex optimization problem over a multi-agent system with Markovian switching communication networks. The objective function is the sum of each agent's local objective function, which cannot be known by other agents. The communication network is assumed to switch over a set of weight-balanced directed graphs with a Markovian property.We propose a consensus sub-gradient algorithm with two time-scale step-sizes to handle the uncertainty due to the Markovian switching topologies and the absence of global gradient information. With a proper selection of step-sizes, we prove the almost sure convergence of all agents' local estimates to the same optimal solution when the union graph of the Markovian network' states is strongly connected and the Markovian network is irreducible. Simulations are given for illustration of the results.