A Markovian Incremental Stochastic Subgradient Algorithm

TL;DR

This paper introduces a stochastic incremental subgradient algorithm that leverages Markov chain-based partial subgradient information for convex function minimization, suitable for networked systems with stochastic information flow.

Contribution

It presents a novel Markovian approach to incremental subgradient methods, allowing for general weighted objectives and flexible information flow modeling.

Findings

Proves convergence to a weighted objective function based on the Markov chain's Cesàro limit.

Allows for non-uniform, general weighting schemes in the optimization process.

Extends previous methods by incorporating stochastic path selection in networks.

Abstract

A stochastic incremental subgradient algorithm for the minimization of a sum of convex functions is introduced. The method sequentially uses partial subgradient information and the sequence of partial subgradients is determined by a general Markov chain. This makes it suitable to be used in networks where the path of information flow is stochastically selected. We prove convergence of the algorithm to a weighted objective function where the weights are given by the Ces\`aro limiting probability distribution of the Markov chain. Unlike previous works in the literature, the Ces\`aro limiting distribution is general (not necessarily uniform), allowing for general weighted objective functions and flexibility in the method.