Analysis of dependence among size, rate and duration in internet flows

TL;DR

This paper rigorously examines the dependence among size, rate, and duration in internet flows using statistical methods, revealing weak correlations and independence especially at extremal values, with implications for network analysis.

Contribution

It introduces a comprehensive statistical analysis of dependence in internet flows, highlighting the limited correlation between size, rate, and duration, and emphasizes the importance of thresholds based on TCP mechanisms.

Findings

Pearson's correlation between size and duration is small.

Correlation between size and rate is generally weak and threshold-dependent.

Extremal dependence analysis shows independence at extreme values.

Abstract

In this paper we examine rigorously the evidence for dependence among data size, transfer rate and duration in Internet flows. We emphasize two statistical approaches for studying dependence, including Pearson's correlation coefficient and the extremal dependence analysis method. We apply these methods to large data sets of packet traces from three networks. Our major results show that Pearson's correlation coefficients between size and duration are much smaller than one might expect. We also find that correlation coefficients between size and rate are generally small and can be strongly affected by applying thresholds to size or duration. Based on Transmission Control Protocol connection startup mechanisms, we argue that thresholds on size should be more useful than thresholds on duration in the analysis of correlations. Using extremal dependence analysis, we draw a similar conclusion, finding remarkable independence for extremal values of size and rate.