Experimental Performance Evaluation of Cloud-Based Analytics-as-a-Service

TL;DR

This paper provides an experimental evaluation of cloud-based Analytics-as-a-Service, analyzing how different storage configurations affect performance and identifying impedance mismatch issues in various service compositions.

Contribution

It introduces a performance evaluation framework for analytic applications on cloud services and proposes a data locality measure to predict performance outcomes.

Findings

Performance varies significantly across storage configurations.

Impedance mismatch causes performance bottlenecks.

Data locality correlates with expected performance.

Abstract

An increasing number of Analytics-as-a-Service solutions has recently seen the light, in the landscape of cloud-based services. These services allow flexible composition of compute and storage components, that create powerful data ingestion and processing pipelines. This work is a first attempt at an experimental evaluation of analytic application performance executed using a wide range of storage service configurations. We present an intuitive notion of data locality, that we use as a proxy to rank different service compositions in terms of expected performance. Through an empirical analysis, we dissect the performance achieved by analytic workloads and unveil problems due to the impedance mismatch that arise in some configurations. Our work paves the way to a better understanding of modern cloud-based analytic services and their performance, both for its end-users and their providers.