Machine Learning for Performance Prediction of Spark Cloud Applications

TL;DR

This paper evaluates machine learning models for predicting Spark application performance on cloud systems, demonstrating improved accuracy over existing methods, especially for irregular workloads and larger datasets.

Contribution

It introduces ML-based performance prediction models that outperform Ernest, a prior ML technique, in accuracy and robustness for diverse Spark workloads.

Findings

ML models reduce prediction error from 126-187% to 5-19%.

ML models perform better on irregular applications and larger datasets.

Our approach matches or exceeds Ernest's performance.

Abstract

Big data applications and analytics are employed in many sectors for a variety of goals: improving customers satisfaction, predicting market behavior or improving processes in public health. These applications consist of complex software stacks that are often run on cloud systems. Predicting execution times is important for estimating the cost of cloud services and for effectively managing the underlying resources at runtime. Machine Learning (ML), providing black box solutions to model the relationship between application performance and system configuration without requiring in-detail knowledge of the system, has become a popular way of predicting the performance of big data applications. We investigate the cost-benefits of using supervised ML models for predicting the performance of applications on Spark, one of today's most widely used frameworks for big data analysis. We compare our approach with \textit{Ernest} (an ML-based technique proposed in the literature by the Spark inventors) on a range of scenarios, application workloads, and cloud system configurations. Our experiments show that Ernest can accurately estimate the performance of very regular applications, but it fails when applications exhibit more irregular patterns and/or when extrapolating on bigger data set sizes. Results show that our models match or exceed Ernest's performance, sometimes enabling us to reduce the prediction error from 126-187% to only 5-19%.