Bias in Internet Measurement Platforms

TL;DR

This paper introduces a comprehensive framework to quantify and analyze biases in Internet measurement platforms caused by non-uniform deployment, providing tools and data to help users understand and mitigate these biases.

Contribution

It presents a generic framework for systematic bias quantification in IMPs and offers open datasets, tools, and visualizations to aid users in bias awareness and mitigation.

Findings

Confirmed known biases in IMPs

Identified less-explored biases

Provided tools for bias visualization and analysis

Abstract

Network operators and researchers frequently use Internet measurement platforms (IMPs), such as RIPE Atlas, RIPE RIS, or RouteViews for, e.g., monitoring network performance, detecting routing events, topology discovery, or route optimization. To interpret the results of their measurements and avoid pitfalls or wrong generalizations, users must understand a platform's limitations. To this end, this paper studies an important limitation of IMPs, the \textit{bias}, which exists due to the non-uniform deployment of the vantage points. Specifically, we introduce a generic framework to systematically and comprehensively quantify the multi-dimensional (e.g., across location, topology, network types, etc.) biases of IMPs. Using the framework and open datasets, we perform a detailed analysis of biases in IMPs that confirms well-known (to the domain experts) biases and sheds light on less-known or unexplored biases. To facilitate IMP users to obtain awareness of and explore bias in their measurements, as well as further research and analyses (e.g., methods for mitigating bias), we publicly share our code and data, and provide online tools (API, Web app, etc.) that calculate and visualize the bias in measurement setups.