On Evaluating Commercial Cloud Services: A Systematic Review
Zheng Li, He Zhang, Liam O'Brien, Rainbow Cai, Shayne, Flint

TL;DR
This systematic review synthesizes current practices in evaluating commercial cloud services, highlighting research gaps and trends to guide future evaluation efforts and improve understanding of cloud service assessment.
Contribution
It provides a comprehensive overview of existing evaluation studies, identifies research gaps, and offers insights into the practical landscape of cloud service evaluation.
Findings
Evaluation of cloud services is chaotic and fragmented.
IaaS is more industry-relevant than PaaS for customers.
Elasticity and Security are long-term evaluation challenges.
Abstract
Background: Cloud Computing is increasingly booming in industry with many competing providers and services. Accordingly, evaluation of commercial Cloud services is necessary. However, the existing evaluation studies are relatively chaotic. There exists tremendous confusion and gap between practices and theory about Cloud services evaluation. Aim: To facilitate relieving the aforementioned chaos, this work aims to synthesize the existing evaluation implementations to outline the state-of-the-practice and also identify research opportunities in Cloud services evaluation. Method: Based on a conceptual evaluation model comprising six steps, the Systematic Literature Review (SLR) method was employed to collect relevant evidence to investigate the Cloud services evaluation step by step. Results: This SLR identified 82 relevant evaluation studies. The overall data collected from these studies…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
