FROST -- Fast row-stochastic optimization with uncoordinated step-sizes

TL;DR

FROST is a distributed optimization algorithm for directed graphs that avoids the need for out-degree knowledge by using row-stochastic weights and uncoordinated step-sizes, achieving linear convergence.

Contribution

FROST introduces a novel approach for distributed optimization on directed graphs without out-degree knowledge, simplifying implementation and ensuring linear convergence.

Findings

FROST converges linearly for smooth, strongly-convex functions.

The algorithm works with uncoordinated, agent-specific step-sizes.

Implementation is straightforward with local weight assignment.

Abstract

In this paper, we discuss distributed optimization over directed graphs, where doubly-stochastic weights cannot be constructed. Most of the existing algorithms overcome this issue by applying push-sum consensus, which utilizes column-stochastic weights. The formulation of column-stochastic weights requires each agent to know (at least) its out-degree, which may be impractical in e.g., broadcast-based communication protocols. In contrast, we describe FROST (Fast Row-stochastic-Optimization with uncoordinated STep-sizes), an optimization algorithm applicable to directed graphs that does not require the knowledge of out-degrees; the implementation of which is straightforward as each agent locally assigns weights to the incoming information and locally chooses a suitable step-size. We show that FROST converges linearly to the optimal solution for smooth and strongly-convex functions given that the largest step-size is positive and sufficiently small.