Impact of Link Failures on the Performance of MapReduce in Data Center Networks

TL;DR

This study uses MILP models to analyze how link failures affect MapReduce shuffling performance across various data center network topologies, revealing significant degradation and identifying the most resilient topology.

Contribution

It introduces a MILP-based approach to quantify link failure impacts on MapReduce performance in different data center network topologies.

Findings

Degradation ranges from 5% to 40% depending on topology.

Server-centric PON-based DCN shows best resilience.

Different topologies respond uniquely to link failures.

Abstract

In this paper, we utilize Mixed Integer Linear Programming (MILP) models to determine the impact of link failures on the performance of shuffling operations in MapReduce when different data center network (DCN) topologies are used. For a set of non-fatal single and multi-links failures, the results indicate that different DCNs experience different completion time degradations ranging between 5% and 40%. The best performance under links failures is achieved by a server-centric PON-based DCN.