Red is Sus: Automated Identification of Low-Quality Service Availability Claims in the US National Broadband Map

TL;DR

This paper presents a machine learning approach to automatically detect low-quality or potentially false broadband availability claims in the US National Broadband Map, improving data reliability for policymakers and the public.

Contribution

It introduces a novel dataset of broadband observations and a high-accuracy classification model to assess the integrity of service provider claims.

Findings

Achieved AUCs over 0.98 in classifying claim accuracy

Developed a dataset with 750k observations from 900+ ISPs

Provided a tool for identifying unreliable broadband data

Abstract

The FCC's National Broadband Map aspires to provide an unprecedented view into broadband availability in the US. However, this map, which also determines eligibility for public grant funding, relies on self-reported data from service providers that in turn have incentives to strategically misrepresent their coverage. In this paper, we develop an approach for automatically identifying these low-quality service claims in the National Broadband Map. To do this, we develop a novel dataset of broadband availability consisting of 750k observations from more than 900 US ISPs, derived from a combination of regulatory data and crowdsourced speed tests. Using this dataset, we develop a model to classify the accuracy of service provider regulatory filings and achieve AUCs over 0.98 for unseen examples. Our approach provides an effective technique to enable policymakers, civil society, and the public to identify portions of the National Broadband Map that are likely to have integrity challenges.