Intelligent Monitoring Framework for Cloud Services: A Data-Driven Approach

TL;DR

This paper presents a data-driven, deep learning framework for recommending cloud service monitors, improving coverage and reducing redundancy by leveraging a large dataset and structured ontology, validated through user studies at Microsoft.

Contribution

It introduces an ontology-based, deep learning approach for automated monitor recommendation in cloud services, addressing ad-hoc and incomplete monitoring practices.

Findings

Achieved high-quality monitor recommendations for most resource classes.

User study rated framework usefulness as 4.27 out of 5.

Derived a structured monitor ontology from over 30,000 monitors.

Abstract

Cloud service owners need to continuously monitor their services to ensure high availability and reliability. Gaps in monitoring can lead to delay in incident detection and significant negative customer impact. Current process of monitor creation is ad-hoc and reactive in nature. Developers create monitors using their tribal knowledge and, primarily, a trial and error based process. As a result, monitors often have incomplete coverage which leads to production issues, or, redundancy which results in noise and wasted effort. In this work, we address this issue by proposing an intelligent monitoring framework that recommends monitors for cloud services based on their service properties. We start by mining the attributes of 30,000+ monitors from 791 production services at Microsoft and derive a structured ontology for monitors. We focus on two crucial dimensions: what to monitor (resources) and which metrics to monitor. We conduct an extensive empirical study and derive key insights on the major classes of monitors employed by cloud services at Microsoft, their associated dimensions, and the interrelationship between service properties and this ontology. Using these insights, we propose a deep learning based framework that recommends monitors based on the service properties. Finally, we conduct a user study with engineers from Microsoft which demonstrates the usefulness of the proposed framework. The proposed framework along with the ontology driven projections, succeeded in creating production quality recommendations for majority of resource classes. This was also validated by the users from the study who rated the framework's usefulness as 4.27 out of 5.