PSSketch: Finding Persistent and Sparse Flow with High Accuracy and Efficiency

TL;DR

PSSketch is a novel high-precision layered sketching method designed to accurately and efficiently identify persistent sparse flows, crucial for early threat detection, outperforming existing solutions in accuracy, memory efficiency, and throughput.

Contribution

The paper introduces PSSketch, a new layered sketch that uses variable-length bitwise counters and an anomaly boundary criterion to improve detection of persistent sparse flows.

Findings

Reduces memory usage by an order of magnitude.

Outperforms state-of-the-art solutions with up to 2.94x higher F1 score.

Achieves higher throughput than existing methods.

Abstract

Finding persistent sparse (PS) flow is critical to early warning of many threats. Previous works have predominantly focused on either heavy or persistent flows, with limited attention given to PS flows. Although some recent studies pay attention to PS flows, they struggle to establish an objective criterion due to insufficient data-driven observations, resulting in reduced accuracy. In this paper, we define a new criterion "anomaly boundary" to distinguish PS flows from regular flows. Specifically, a flow whose persistence exceeds a threshold will be protected, while a protected flow with a density lower than a threshold is reported as a PS flow. We then introduce PSSketch, a high-precision layered sketch to find PS flows. PSSketch employs variable-length bitwise counters, where the first layer tracks the frequency and persistence of all flows, and the second layer protects potential PS flows and records overflow counts from the first layer. Some optimizations have also been implemented to reduce memory consumption further and improve accuracy. The experiments show that PSSketch reduces memory consumption by an order of magnitude compared to the strawman solution combined with existing work. Compared with SOTA solutions for finding PS flows, it outperforms up to 2.94x in F1 score and reduces ARE by 1-2 orders of magnitude. Meanwhile, PSSketch achieves a higher throughput than these solutions.