RLC Circuits based Distributed Mirror Descent Method

TL;DR

This paper introduces a novel distributed mirror descent algorithm inspired by RLC circuits, achieving similar convergence rates as existing methods but with reduced computational cost, and extends it to noisy and composite objectives.

Contribution

The paper proposes a new RLC circuit-inspired distributed mirror descent method with improved efficiency and broader applicability to noisy and composite optimization problems.

Findings

Achieves (1/k) convergence rate with half the computation per iteration.

Extends the method to stochastic gradient noise scenarios.

Demonstrates effectiveness through numerical experiments.

Abstract

We consider distributed optimization with smooth convex objective functions defined on an undirected connected graph. Inspired by mirror descent mehod and RLC circuits, we propose a novel distributed mirror descent method. Compared with mirror-prox method, our algorithm achieves the same \(\mathcal{O}(1/k)\) iteration complexity with only half the computation cost per iteration. We further extend our results to cases where a) gradients are corrupted by stochastic noise, and b) objective function is composed of both smooth and non-smooth terms. We demonstrate our theoretical results via numerical experiments.