Hierarchical coded elastic computing

TL;DR

This paper introduces two hierarchical coded elastic computing schemes that leverage elasticity and stragglers in cloud systems to significantly reduce computation time, demonstrating a 45% improvement over existing methods.

Contribution

The paper proposes novel hierarchical coded elastic computing schemes that enhance task allocation and speed up computations in elastic cloud environments.

Findings

45% reduction in average finishing time

Effective utilization of elastic nodes and stragglers

Improved task scheduling efficiency

Abstract

Elasticity is offered by cloud service providers to exploit under-utilized computing resources. The low-cost elastic nodes can leave and join any time during the computation cycle. The possibility of elastic events occurring together with the problem of slow nodes, referred to as stragglers, increases the uncertainty of the system, leading to computation delay. Recent results have shown that coded computing can be used to reduce the negative effect of elasticity and stragglers. In this paper, we propose two hierarchical coded elastic computing schemes that can further speed up the system by exploiting stragglers and effectively allocating tasks among available nodes. In our simulations, our scheme realizes 45% improvement in average finishing time compared to the state-of-the-art coded elastic computing scheme.