Assessing the Impact of Case Correction Methods on the Fairness of COVID-19 Predictive Models

TL;DR

This study evaluates how different COVID-19 case correction methods affect the fairness of predictive models, revealing that some methods can reduce bias while others may increase disparities among racial and socioeconomic groups.

Contribution

The paper introduces an auditing approach to assess the fairness impact of case correction methods on COVID-19 prediction models, highlighting potential biases introduced by these methods.

Findings

One correction method improved fairness by reducing performance disparities.

Another correction method increased bias, worsening disparities.

Correction methods can either mitigate or exacerbate existing biases.

Abstract

One of the central difficulties of addressing the COVID-19 pandemic has been accurately measuring and predicting the spread of infections. In particular, official COVID-19 case counts in the United States are under counts of actual caseloads due to the absence of universal testing policies. Researchers have proposed a variety of methods for recovering true caseloads, often through the estimation of statistical models on more reliable measures, such as death and hospitalization counts, positivity rates, and demographics. However, given the disproportionate impact of COVID-19 on marginalized racial, ethnic, and socioeconomic groups, it is important to consider potential unintended effects of case correction methods on these groups. Thus, we investigate two of these correction methods for their impact on a downstream COVID-19 case prediction task. For that purpose, we tailor an auditing approach and evaluation protocol to analyze the fairness of the COVID-19 prediction task by measuring the difference in model performance between majority-White counties and majority-minority counties. We find that one of the correction methods improves fairness, decreasing differences in performance between majority-White and majority-minority counties, while the other method increases differences, introducing bias. While these results are mixed, it is evident that correction methods have the potential to exacerbate existing biases in COVID-19 case data and in downstream prediction tasks. Researchers planning to develop or use case correction methods must be careful to consider negative effects on marginalized groups.