DocLite: A Docker-Based Lightweight Cloud Benchmarking Tool

TL;DR

DocLite is a Docker-based lightweight benchmarking tool that enables rapid, near real-time performance ranking of VMs by benchmarking small containerized portions instead of entire VMs, significantly reducing benchmarking time.

Contribution

This paper introduces DocLite, a novel Docker container-based benchmarking approach that significantly speeds up VM performance evaluation with minimal accuracy loss.

Findings

Up to 91 times faster than full VM benchmarking

Achieves over 90% accuracy in performance ranking

Small containers provide comparable rankings to large containers

Abstract

Existing benchmarking methods are time consuming processes as they typically benchmark the entire Virtual Machine (VM) in order to generate accurate performance data, making them less suitable for real-time analytics. The research in this paper is aimed to surmount the above challenge by presenting DocLite - Docker Container-based Lightweight benchmarking tool. DocLite explores lightweight cloud benchmarking methods for rapidly executing benchmarks in near real-time. DocLite is built on the Docker container technology, which allows a user-defined memory size and number of CPU cores of the VM to be benchmarked. The tool incorporates two benchmarking methods - the first referred to as the native method employs containers to benchmark a small portion of the VM and generate performance ranks, and the second uses historic benchmark data along with the native method as a hybrid to generate VM ranks. The proposed methods are evaluated on three use-cases and are observed to be up to 91 times faster than benchmarking the entire VM. In both methods, small containers provide the same quality of rankings as a large container. The native method generates ranks with over 90% and 86% accuracy for sequential and parallel execution of an application compared against benchmarking the whole VM. The hybrid method did not improve the quality of the rankings significantly.