DRAM Failure Prediction in AIOps: Empirical Evaluation, Challenges and Opportunities

TL;DR

This paper empirically evaluates machine learning techniques for DRAM failure prediction in large-scale data centers, highlighting challenges and future research opportunities using a large multi-source dataset.

Contribution

It provides a comprehensive empirical comparison of classifiers and anomaly detectors for DRAM failure prediction using a large, multi-source dataset from Alibaba Cloud.

Findings

Multi-class classification achieves high accuracy in failure prediction.

Unsupervised anomaly detection offers alternative insights.

Challenges include data imbalance and feature heterogeneity.

Abstract

DRAM failure prediction is a vital task in AIOps, which is crucial to maintain the reliability and sustainable service of large-scale data centers. However, limited work has been done on DRAM failure prediction mainly due to the lack of public available datasets. This paper presents a comprehensive empirical evaluation of diverse machine learning techniques for DRAM failure prediction using a large-scale multi-source dataset, including more than three millions of records of kernel, address, and mcelog data, provided by Alibaba Cloud through PAKDD 2021 competition. Particularly, we first formulate the problem as a multi-class classification task and exhaustively evaluate seven popular/state-of-the-art classifiers on both the individual and multiple data sources. We then formulate the problem as an unsupervised anomaly detection task and evaluate three state-of-the-art anomaly detectors. Further, based on the empirical results and our experience of attending this competition, we discuss major challenges and present future research opportunities in this task.