Mitigating Interference of Microservices with a Scoring Mechanism in Large-scale Clusters

TL;DR

This paper introduces PISM, a proactive framework that models and scores interference from best-effort jobs to latency-critical services, optimizing scheduling to significantly reduce interference and improve performance in large-scale clusters.

Contribution

PISM is a novel data-driven framework that characterizes BEJs, models their impact on LCS response times, and uses interference scoring for optimized scheduling.

Findings

PISM reduces cluster interference by up to 41.5%.

PISM improves long-tail LCS throughput by 76.4%.

Effective BEJ scheduling mitigates interference in large-scale clusters.

Abstract

Co-locating latency-critical services (LCSs) and best-effort jobs (BEJs) constitute the principal approach for enhancing resource utilization in production. Nevertheless, the co-location practice hurts the performance of LCSs due to resource competition, even when employing isolation technology. Through an extensive analysis of voluminous real trace data derived from two production clusters, we observe that BEJs typically exhibit periodic execution patterns and serve as the primary sources of interference to LCSs. Furthermore, despite occupying the same level of resource consumption, the diverse compositions of BEJs can result in varying degrees of interference on LCSs. Subsequently, we propose PISM, a proactive Performance Interference Scoring and Mitigating framework for LCSs through the optimization of BEJ scheduling. Firstly, PISM adopts a data-driven approach to establish a characterization and classification methodology for BEJs. Secondly, PISM models the relationship between the composition of BEJs on servers and the response time (RT) of LCSs. Thirdly, PISM establishes an interference scoring mechanism in terms of RT, which serves as the foundation for BEJ scheduling. We assess the effectiveness of PISM on a small-scale cluster and through extensive data-driven simulations. The experiment results demonstrate that PISM can reduce cluster interference by up to 41.5%, and improve the throughput of long-tail LCSs by 76.4%.