Communication-Aware Scheduling of Serial Tasks for Dispersed Computing

TL;DR

This paper addresses the challenge of scheduling serial tasks in dispersed computing networks by considering communication delays, proposing a capacity characterization, a novel virtual queue model, and a throughput-optimal scheduling policy.

Contribution

It introduces a new model for communication-aware task scheduling in dispersed networks, including capacity analysis and a Max-Weight based policy.

Findings

Characterized the capacity region of dispersed computing networks.

Developed a virtual queueing network model for the system.

Proposed a throughput-optimal scheduling policy based on Lyapunov analysis.

Abstract

There is a growing interest in development of in-network dispersed computing paradigms that leverage the computing capabilities of heterogeneous resources dispersed across the network for processing massive amount of data is collected at the edge of the network. We consider the problem of task scheduling for such networks, in a dynamic setting in which arriving computation jobs are modeled as chains, with nodes representing tasks, and edges representing precedence constraints among tasks. In our proposed model, motivated by significant communication costs in dispersed computing environments, the communication times are taken into account. More specifically, we consider a network where servers are capable of serving all task types, and sending the results of processed tasks from one server to another server results in some communication delay that makes the design of optimal scheduling policy significantly more challenging than classical queueing networks. As the main contributions of the paper, we first characterize the capacity region of the network, then propose a novel virtual queueing network encoding the state of the network. Finally, we propose a Max-Weight type scheduling policy, and considering the virtual queueing network in the fluid limit, we use a Lyapunov argument to show that the policy is throughput-optimal.