DC-Prophet: Predicting Catastrophic Machine Failures in DataCenters

TL;DR

This paper introduces DC-Prophet, a machine learning framework that predicts catastrophic server failures in data centers with high accuracy, enabling proactive maintenance and improved reliability.

Contribution

The paper presents a novel two-stage prediction framework combining One-Class SVM and Random Forest, achieving state-of-the-art failure prediction performance on large-scale data center traces.

Findings

Achieves 0.93 AUC in failure prediction

Attains 0.88 F3-score, outperforming classical methods by 39.45%

Effectively categorizes and predicts three types of catastrophic failures

Abstract

When will a server fail catastrophically in an industrial datacenter? Is it possible to forecast these failures so preventive actions can be taken to increase the reliability of a datacenter? To answer these questions, we have studied what are probably the largest, publicly available datacenter traces, containing more than 104 million events from 12,500 machines. Among these samples, we observe and categorize three types of machine failures, all of which are catastrophic and may lead to information loss, or even worse, reliability degradation of a datacenter. We further propose a two-stage framework-DC-Prophet-based on One-Class Support Vector Machine and Random Forest. DC-Prophet extracts surprising patterns and accurately predicts the next failure of a machine. Experimental results show that DC-Prophet achieves an AUC of 0.93 in predicting the next machine failure, and a F3-score of 0.88 (out of 1). On average, DC-Prophet outperforms other classical machine learning methods by 39.45% in F3-score.