Comparing Broadband ISP Performance using Big Data from M-Lab
Xiaohong Deng, Yun Feng, Thanchanok Sutjarittham, Hassan, Habibi Gharakheili, Blanca Gallego, Vijay Sivaraman

TL;DR
This paper develops a causal inference-based framework to compare broadband ISP performance using large-scale M-Lab data, addressing biases and revealing more accurate performance differences.
Contribution
It introduces a novel data preprocessing and visualization tool, applies multi-variate matching to reduce bias, and refines performance estimation methods for unbiased ISP comparison.
Findings
ISPs are more similar in performance than previously thought.
Biases due to subscriber attributes significantly affect performance comparisons.
Refined models provide more accurate ISP rankings.
Abstract
Comparing ISPs on broadband speed is challenging, since measurements can vary due to subscriber attributes such as operation system and test conditions such as access capacity, server distance, TCP window size, time-of-day, and network segment size. In this paper, we draw inspiration from observational studies in medicine, which face a similar challenge in comparing the effect of treatments on patients with diverse characteristics, and have successfully tackled this using "causal inference" techniques for {\em post facto} analysis of medical records. Our first contribution is to develop a tool to pre-process and visualize the millions of data points in M-Lab at various time- and space-granularities to get preliminary insights on factors affecting broadband performance. Next, we analyze 24 months of data pertaining to twelve ISPs across three countries, and demonstrate that there is…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Causal Inference Techniques · Healthcare Policy and Management
