A Factor Framework for Experimental Design for Performance Evaluation of Commercial Cloud Services
Zheng Li, Liam O'Brien, He Zhang, Rainbow Cai

TL;DR
This paper introduces a systematic factor framework to guide the design of performance evaluation experiments for commercial Cloud services, addressing the lack of structured approaches in current practices.
Contribution
It proposes a comprehensive factor framework based on prior taxonomy and modeling work to improve experimental design in Cloud service performance evaluation.
Findings
Capsules current evaluation factors in Cloud computing
Facilitates systematic experimental design
Supports better decision-making for Cloud service evaluation
Abstract
Given the diversity of commercial Cloud services, performance evaluations of candidate services would be crucial and beneficial for both service customers (e.g. cost-benefit analysis) and providers (e.g. direction of service improvement). Before an evaluation implementation, the selection of suitable factors (also called parameters or variables) plays a prerequisite role in designing evaluation experiments. However, there seems a lack of systematic approaches to factor selection for Cloud services performance evaluation. In other words, evaluators randomly and intuitively concerned experimental factors in most of the existing evaluation studies. Based on our previous taxonomy and modeling work, this paper proposes a factor framework for experimental design for performance evaluation of commercial Cloud services. This framework capsules the state-of-the-practice of performance evaluation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
