Detecting and Ranking Causal Anomalies in End-to-End Complex System

TL;DR

This paper introduces RCAE2E, a novel framework for detecting and ranking causal anomalies in complex systems, addressing limitations of traditional correlation-based methods by considering system states and time-lags, validated on synthetic and real data.

Contribution

The paper proposes RCAE2E, a new framework that improves causal anomaly detection by incorporating system state diversity and time-lag considerations, overcoming ARX model limitations.

Findings

RCAE2E effectively detects causal anomalies in synthetic data.

RCAE2E successfully identifies anomalies in real-world factory data.

The method outperforms traditional correlation-based approaches.

Abstract

With the rapid development of technology, the automated monitoring systems of large-scale factories are becoming more and more important. By collecting a large amount of machine sensor data, we can have many ways to find anomalies. We believe that the real core value of an automated monitoring system is to identify and track the cause of the problem. The most famous method for finding causal anomalies is RCA, but there are many problems that cannot be ignored. They used the AutoRegressive eXogenous (ARX) model to create a time-invariant correlation network as a machine profile, and then use this profile to track the causal anomalies by means of a method called fault propagation. There are two major problems in describing the behavior of a machine by using the correlation network established by ARX: (1) It does not take into account the diversity of states (2) It does not separately consider the correlations with different time-lag. Based on these problems, we propose a framework called Ranking Causal Anomalies in End-to-End System (RCAE2E), which completely solves the problems mentioned above. In the experimental part, we use synthetic data and real-world large-scale photoelectric factory data to verify the correctness and existence of our method hypothesis.