Diffusion-based Time Series Data Imputation for Microsoft 365

TL;DR

This paper introduces Diffusion+, a diffusion-based model designed to improve data quality in cloud systems by efficiently imputing missing time series data, thereby enhancing failure prediction accuracy in Microsoft 365.

Contribution

The paper proposes a novel diffusion-based data imputation method, Diffusion+, specifically tailored for large-scale cloud system data, improving failure prediction performance.

Findings

Diffusion+ effectively imputes missing data in cloud system time series.

Imputation with Diffusion+ improves downstream failure prediction accuracy.

The method demonstrates sample efficiency and practical applicability.

Abstract

Reliability is extremely important for large-scale cloud systems like Microsoft 365. Cloud failures such as disk failure, node failure, etc. threaten service reliability, resulting in online service interruptions and economic loss. Existing works focus on predicting cloud failures and proactively taking action before failures happen. However, they suffer from poor data quality like data missing in model training and prediction, which limits the performance. In this paper, we focus on enhancing data quality through data imputation by the proposed Diffusion+, a sample-efficient diffusion model, to impute the missing data efficiently based on the observed data. Our experiments and application practice show that our model contributes to improving the performance of the downstream failure prediction task.