# Most memory efficient distributed super points detection on core networks

## TL;DR

This paper introduces a highly memory-efficient distributed method for detecting super points in core networks, combining novel algorithms to achieve high accuracy with minimal memory usage and suitability for parallel processing.

## Contribution

It presents the most memory-efficient super points detection scheme with a novel combination of short and long estimators, optimized for parallel GPU implementation.

## Key findings

- Achieves highest accuracy with less than one-fifth memory of existing algorithms.
- No data conflict or floating operations, enabling parallel processing.
- Effective extension to sliding time windows.

## Abstract

The super point, a host which communicates with lots of others, is a kind of special hosts gotten great focus. Mining super point at the edge of a network is the foundation of many network research fields. In this paper, we proposed the most memory efficient super points detection scheme. This scheme contains a super points reconstruction algorithm called short estimator and a super points filter algorithm called long estimator. Short estimator gives a super points candidate list using thousands of bytes memory and long estimator improves the accuracy of detection result using millions of bytes memory. Combining short estimator and long estimator, our scheme acquires the highest accuracy using the smallest memory than other algorithms. There is no data conflict and floating operation in our scheme. This ensures that our scheme is suitable for parallel running and we deploy our scheme on a common GPU to accelerate processing speed. We also describe how to extend our algorithm to sliding time. Experiments on several real-world core network traffics show that our algorithm acquires the highest accuracy with only consuming littler than one-fifth memory of other algorithms.