Rating Sentiment Analysis Systems for Bias through a Causal Lens

TL;DR

This paper proposes a causal perturbation method to evaluate sentiment analysis systems for bias, providing a rating system to measure their robustness and fairness concerning protected attributes like gender or race.

Contribution

It introduces a novel causal perturbation approach to assess and rate SASs for bias, enabling comparison without access to model internals.

Findings

Effective in detecting bias sensitivity in SASs

Provides a fine-grained rating system for robustness

Facilitates comparison of different SASs based on bias

Abstract

Sentiment Analysis Systems (SASs) are data-driven Artificial Intelligence (AI) systems that, given a piece of text, assign one or more numbers conveying the polarity and emotional intensity expressed in the input. Like other automatic machine learning systems, they have also been known to exhibit model uncertainty where a (small) change in the input leads to drastic swings in the output. This can be especially problematic when inputs are related to protected features like gender or race since such behavior can be perceived as a lack of fairness, i.e., bias. We introduce a novel method to assess and rate SASs where inputs are perturbed in a controlled causal setting to test if the output sentiment is sensitive to protected variables even when other components of the textual input, e.g., chosen emotion words, are fixed. We then use the result to assign labels (ratings) at fine-grained and overall levels to convey the robustness of the SAS to input changes. The ratings serve as a principled basis to compare SASs and choose among them based on behavior. It benefits all users, especially developers who reuse off-the-shelf SASs to build larger AI systems but do not have access to their code or training data to compare.