Simultaneous Monitoring of a Large Number of Heterogeneous Categorical Data Streams

TL;DR

This paper introduces a robust and efficient method for simultaneously monitoring numerous heterogeneous categorical data streams, including nominal and ordinal types, using a normalized global goodness-of-fit statistic.

Contribution

It develops a novel monitoring scheme that integrates local likelihood ratio tests for diverse categorical streams into a unified global statistic, addressing high dimensionality.

Findings

Method demonstrates robustness in simulations

Effective for high-dimensional heterogeneous data

Outperforms traditional control charts

Abstract

This article proposes a powerful scheme to monitor a large number of categorical data streams with heterogeneous parameters or nature. The data streams considered may be either nominal with a number of attribute levels or ordinal with some natural order among their attribute levels, such as good, marginal, and bad. For an ordinal data stream, it is assumed that there is a corresponding latent continuous data stream determining it. Furthermore, different data streams may have different number of attribute levels and different values of level probabilities. Due to high dimensionality, traditional multivariate categorical control charts cannot be applied. Here we integrate the local exponentially weighted likelihood ratio test statistics from each single stream, regardless of nominal or ordinal, into a powerful goodness-of-fit test by some normalization procedure. A global monitoring statistic is proposed ultimately. Simulation results have demonstrated the robustness and efficiency of our method.