Asynchronous Optimization over Weakly Coupled Renewal Systems

TL;DR

This paper develops a distributed asynchronous optimization algorithm for coupled renewal systems, achieving near-optimal solutions with provable convergence guarantees, applicable to networks and decision processes.

Contribution

It introduces a novel distributed algorithm for asynchronous optimization over coupled renewal systems with convergence and constraint satisfaction guarantees.

Findings

Achieves $ ext{O}(rac{1}{ ext{ extit{varepsilon}}^2})$ convergence time.

Ensures satisfaction of time average resource constraints.

Attains $ ext{O}( ext{ extit{varepsilon}})$ near optimality.

Abstract

This paper considers optimization over multiple renewal systems coupled by time average constraints. These systems act asynchronously over variable length frames. For each system, at the beginning of each renewal frame, it chooses an action which affects the duration of its own frame, the penalty, and the resource expenditure throughout the frame. The goal is to minimize the overall time average penalty subject to several overall time average resource constraints which couple these systems. This problem has applications to task processing networks, coupled Markov decision processes(MDPs) and so on. We propose a distributed algorithm so that each system can make its own decision after observing a global multiplier which is updated slot-wise. We show that this algorithm satisfies the desired constraints and achieves $\mathcal{O}(\varepsilon)$ near optimality with $\mathcal{O}(1/\varepsilon^2)$ convergence time.