Data Volume-aware Computation Task Scheduling for Smart Grid Data Analytic Applications

TL;DR

This paper proposes a novel integrated scheduling scheme for smart grid data analytics that considers data volume and transmission order to reduce delays and improve response times in large-scale distributed systems.

Contribution

It introduces a topology-aware Branch and Cut optimization method for efficient scheduling of computation and data transfer in smart grid analytics.

Findings

The proposed method effectively reduces data transfer delays.

It outperforms existing scheduling algorithms in simulation.

The approach adapts to complex job graph topologies.

Abstract

Emerging smart grid applications analyze large amounts of data collected from millions of meters and systems to facilitate distributed monitoring and real-time control tasks. However, current parallel data processing systems are designed for common applications, unaware of the massive volume of the collected data, causing long data transfer delay during the computation and slow response time of smart grid systems. A promising direction to reduce delay is to jointly schedule computation tasks and data transfers. We identify that the smart grid data analytic jobs require the intermediate data among different computation stages to be transmitted orderly to avoid network congestion. This new feature prevents current scheduling algorithms from being efficient. In this work, an integrated computing and communication task scheduling scheme is proposed. The mathematical formulation of smart grid data analytic jobs scheduling problem is given, which is unsolvable by existing optimization methods due to the strongly coupled constraints. Several techniques are combined to linearize it for adapting the Branch and Cut method. Based on the topological information in the job graph, the Topology Aware Branch and Cut method is further proposed to speed up searching for optimal solutions. Numerical results demonstrate the effectiveness of the proposed method.