BatchLens: A Visualization Approach for Analyzing Batch Jobs in Cloud Systems

TL;DR

BatchLens is an interactive visualization tool designed to help cloud system users and providers analyze batch job performance and identify anomalies in complex workflows using real-time data and visual analytics.

Contribution

We introduce BatchLens, a novel visual analytics approach that enables intuitive exploration and root-cause analysis of batch job behaviors in cloud systems.

Findings

Effective identification of anomalous behaviors in cloud batch jobs

Improved understanding of system performance metrics in real time

Successful case study on Alibaba workload trace datasets

Abstract

Cloud systems are becoming increasingly powerful and complex. It is highly challenging to identify anomalous execution behaviors and pinpoint problems by examining the overwhelming intermediate results/states in complex application workflows. Domain scientists urgently need a friendly and functional interface to understand the quality of the computing services and the performance of their applications in real time. To meet these needs, we explore data generated by job schedulers and investigate general performance metrics (e.g., utilization of CPU, memory and disk I/O). Specifically, we propose an interactive visual analytics approach, BatchLens, to provide both providers and users of cloud service with an intuitive and effective way to explore the status of system batch jobs and help them conduct root-cause analysis of anomalous behaviors in batch jobs. We demonstrate the effectiveness of BatchLens through a case study on the public Alibaba bench workload trace datasets.