Leveraging Administrative Data for Bias Audits: Assessing Disparate Coverage with Mobility Data for COVID-19 Policy

TL;DR

This paper demonstrates how linking administrative data, like voter records, can reveal demographic biases in mobility data used for COVID-19 policies, highlighting underrepresentation of vulnerable groups.

Contribution

It introduces a novel method for bias auditing of mobility data by linking it with voter records to assess demographic coverage without ground truth labels.

Findings

Mobility data underrepresents older and non-white voters.

Bias in mobility data could lead to inequitable public health resource allocation.

Linking administrative data enables bias detection without demographic labels.

Abstract

Anonymized smartphone-based mobility data has been widely adopted in devising and evaluating COVID-19 response strategies such as the targeting of public health resources. Yet little attention has been paid to measurement validity and demographic bias, due in part to the lack of documentation about which users are represented as well as the challenge of obtaining ground truth data on unique visits and demographics. We illustrate how linking large-scale administrative data can enable auditing mobility data for bias in the absence of demographic information and ground truth labels. More precisely, we show that linking voter roll data -- containing individual-level voter turnout for specific voting locations along with race and age -- can facilitate the construction of rigorous bias and reliability tests. These tests illuminate a sampling bias that is particularly noteworthy in the pandemic context: older and non-white voters are less likely to be captured by mobility data. We show that allocating public health resources based on such mobility data could disproportionately harm high-risk elderly and minority groups.