Data Integrity Error Localization in Networked Systems with Missing Data

TL;DR

This paper introduces a novel multi-output prediction model for network failure localization in wide-area networks, effectively handling missing data through multivariate imputation, and demonstrating significant performance improvements.

Contribution

It presents the first application of imputation techniques to address missing data in network failure localization, enhancing accuracy in wide-area network diagnosis.

Findings

Multivariate imputation improves failure localization accuracy.

The model effectively handles incomplete measurement data.

Significant performance gains with selected imputation algorithms.

Abstract

Most recent network failure diagnosis systems focused on data center networks where complex measurement systems can be deployed to derive routing information and ensure network coverage in order to achieve accurate and fast fault localization. In this paper, we target wide-area networks that support data-intensive distributed applications. We first present a new multi-output prediction model that directly maps the application level observations to localize the system component failures. In reality, this application-centric approach may face the missing data challenge as some input (feature) data to the inference models may be missing due to incomplete or lost measurements in wide area networks. We show that the presented prediction model naturally allows the {\it multivariate} imputation to recover the missing data. We evaluate multiple imputation algorithms and show that the prediction performance can be improved significantly in a large-scale network. As far as we know, this is the first study on the missing data issue and applying imputation techniques in network failure localization.