Measurement-based Resource Allocation and Control in Data Centers: A Survey

TL;DR

This survey reviews measurement-based network resource allocation and control in data centers, emphasizing techniques for ensuring application performance guarantees amidst network interference, and discusses recent advances including machine learning approaches.

Contribution

It provides a comprehensive overview of network measurement techniques, traffic characteristics, monitoring evolution, and scheduling frameworks for resource allocation in data centers.

Findings

Network monitoring has evolved from SDN to programmable dataplanes.

Measurement-based approaches can improve resource allocation accuracy.

Machine learning is emerging as a key tool for resource control.

Abstract

Data centers have become ubiquitous for today's businesses. From banks to startups, they rely on cloud infrastructure to deploy user applications. In this context, it is vital to provide users with application performance guarantees. Network interference is one of the causes of unpredictable application performance, and many solutions have been proposed over the years. The main objective of this survey is to familiarize the reader with research into network measurement-based resource allocation and control in data centers, focusing on network resources in order to provide cloud performance guarantees. We start with a primer on general network measurement techniques and data center network and applications to give the reader context. We then summarize the characteristics of network traffic and cluster workloads in data centers, which are pivotal for measurement-based allocation and control. We study and compare network monitoring in data centers, giving an overview on their evolution from Software-Defined Networking (SDN) to programmable dataplanes-based. The network monitoring information can serve as input to cluster allocation and scheduling decisions. We next categorize cluster scheduling frameworks, and perform an analysis of those that provide network guarantees in data centers, and we also look at emergent Machine Learning-driven resource allocation and control. We conclude with a discussion about future research directions.