Evaluating Pre-Training Bias on Severe Acute Respiratory Syndrome Dataset

TL;DR

This paper investigates how pre-training bias metrics vary across regions in Brazil using the SARS dataset, aiming to identify potential biases before model training and deployment.

Contribution

It introduces a visualization approach for pre-training bias metrics across regions, highlighting regional differences in bias and model performance.

Findings

Bias metrics vary significantly across regions.

Model performance correlates with bias metric values.

Visualization helps identify potential harm before training.

Abstract

Machine learning (ML) is a growing field of computer science that has found many practical applications in several domains, including Health. However, as data grows in size and availability, and the number of models that aim to aid or replace human decisions, it raises the concern that these models can be susceptible to bias, which can lead to harm to specific individuals by basing its decisions on protected attributes such as gender, religion, sexual orientation, ethnicity, and others. Visualization techniques might generate insights and help summarize large datasets, enabling data scientists to understand the data better before training a model by evaluating pre-training metrics applied to the datasets before training, which might contribute to identifying potential harm before any effort is put into training and deploying the models. This work uses the severe acute respiratory syndrome dataset from OpenDataSUS to visualize three pre-training bias metrics and their distribution across different regions in Brazil. A random forest model is trained in each region and applied to the others. The aim is to compare the bias for the different regions, focusing on their protected attributes and comparing the model's performance with the metric values.