Improved Algorithms for Distributed Entropy Monitoring

TL;DR

This paper presents improved algorithms for monitoring entropy functions in distributed data systems, enhancing efficiency and implementing AMS sampling in a distributed context for network monitoring applications.

Contribution

It introduces new algorithms that outperform previous methods for distributed entropy monitoring and adapts AMS sampling for distributed systems, broadening its applicability.

Findings

Enhanced algorithms for entropy monitoring with lower communication costs

Successful implementation of AMS sampling in distributed environments

Improved detection capabilities in network monitoring scenarios

Abstract

Modern data management systems often need to deal with massive, dynamic and inherently distributed data sources. We collect the data using a distributed network, and at the same time try to maintain a global view of the data at a central coordinator using a minimal amount of communication. Such applications have been captured by the distributed monitoring model which has attracted a lot of attention in recent years. In this paper we investigate the monitoring of the entropy functions, which are very useful in network monitoring applications such as detecting distributed denial-of-service attacks. Our results improve the previous best results by Arackaparambil et al. [2]. Our technical contribution also includes implementing the celebrated AMS sampling method (by Alon et al. [1]) in the distributed monitoring model, which could be of independent interest.