Enhancing Robustness of On-line Learning Models on Highly Noisy Data

TL;DR

This paper introduces a robust online anomaly detection framework, RAD, that effectively handles highly noisy data across various applications by continuously cleansing data and leveraging ensemble predictions.

Contribution

The paper proposes a novel ensemble-based online data selection framework, RAD, with extensions for adaptive learning and oracle knowledge, improving robustness against noisy labels in anomaly detection.

Findings

Achieves up to 98.95% accuracy in IoT attack detection under 40% label noise.

Improves cloud task failure prediction accuracy by 14% under 40% label noise.

Reaches 77.51% accuracy in face recognition under 30% label noise.

Abstract

Classification algorithms have been widely adopted to detect anomalies for various systems, e.g., IoT, cloud and face recognition, under the common assumption that the data source is clean, i.e., features and labels are correctly set. However, data collected from the wild can be unreliable due to careless annotations or malicious data transformation for incorrect anomaly detection. In this paper, we extend a two-layer on-line data selection framework: Robust Anomaly Detector (RAD) with a newly designed ensemble prediction where both layers contribute to the final anomaly detection decision. To adapt to the on-line nature of anomaly detection, we consider additional features of conflicting opinions of classifiers, repetitive cleaning, and oracle knowledge. We on-line learn from incoming data streams and continuously cleanse the data, so as to adapt to the increasing learning capacity from the larger accumulated data set. Moreover, we explore the concept of oracle learning that provides additional information of true labels for difficult data points. We specifically focus on three use cases, (i) detecting 10 classes of IoT attacks, (ii) predicting 4 classes of task failures of big data jobs, and (iii) recognising 100 celebrities faces. Our evaluation results show that RAD can robustly improve the accuracy of anomaly detection, to reach up to 98.95% for IoT device attacks (i.e., +7%), up to 85.03% for cloud task failures (i.e., +14%) under 40% label noise, and for its extension, it can reach up to 77.51% for face recognition (i.e., +39%) under 30% label noise. The proposed RAD and its extensions are general and can be applied to different anomaly detection algorithms.