Fishing in Poisson streams: focusing on the whales, ignoring the minnows

TL;DR

This paper introduces a low-complexity compressed sensing method using expander graphs to efficiently reconstruct packet arrival rates and counts in Poisson streams, enabling accurate network traffic analysis with minimal counters.

Contribution

It presents a novel approach combining expander graph-based compressed sensing with Poisson process modeling for network traffic measurement.

Findings

Efficient reconstruction of packet counts and rates using sparse binary matrices.

Comparison of two estimation methods for counts and rates.

Accurate recovery with reduced computational complexity.

Abstract

This paper describes a low-complexity approach for reconstructing average packet arrival rates and instantaneous packet counts at a router in a communication network, where the arrivals of packets in each flow follow a Poisson process. Assuming that the rate vector of this Poisson process is sparse or approximately sparse, the goal is to maintain a compressed summary of the process sample paths using a small number of counters, such that at any time it is possible to reconstruct both the total number of packets in each flow and the underlying rate vector. We show that these tasks can be accomplished efficiently and accurately using compressed sensing with expander graphs. In particular, the compressive counts are a linear transformation of the underlying counting process by the adjacency matrix of an unbalanced expander. Such a matrix is binary and sparse, which allows for efficient incrementing when new packets arrive. We describe, analyze, and compare two methods that can be used to estimate both the current vector of total packet counts and the underlying vector of arrival rates.