Cloud Failure Prediction with Hierarchical Temporal Memory: An Empirical Assessment

TL;DR

This paper systematically evaluates Hierarchical Temporal Memory (HTM) for online failure prediction in cloud systems, demonstrating its effectiveness as an unsupervised, continuously learning anomaly detection method with promising results.

Contribution

First empirical assessment of HTM's capability for failure prediction in cloud environments, highlighting its practicality and effectiveness.

Findings

HTM achieved an F-measure of 0.76 in failure prediction.

HTM performed well across 12 fault types in cloud systems.

HTM offers a viable unsupervised alternative to supervised failure prediction methods.

Abstract

Hierarchical Temporal Memory (HTM) is an unsupervised learning algorithm inspired by the features of the neocortex that can be used to continuously process stream data and detect anomalies, without requiring a large amount of data for training nor requiring labeled data. HTM is also able to continuously learn from samples, providing a model that is always up-to-date with respect to observations. These characteristics make HTM particularly suitable for supporting online failure prediction in cloud systems, which are systems with a dynamically changing behavior that must be monitored to anticipate problems. This paper presents the first systematic study that assesses HTM in the context of failure prediction. The results that we obtained considering 72 configurations of HTM applied to 12 different types of faults introduced in the Clearwater cloud system show that HTM can help to predict failures with sufficient effectiveness (F-measure = 0.76), representing an interesting practical alternative to (semi-)supervised algorithms.