Auditing Gender Analyzers on Text Data

TL;DR

This paper audits existing gender analyzers and ChatGPT for biases against non-binary individuals, revealing significant inaccuracies and societal biases, and demonstrates that fine-tuned BERT models on inclusive datasets can mitigate these biases effectively.

Contribution

The study provides a comprehensive bias audit of popular gender analyzers and ChatGPT, and introduces a fine-tuned BERT approach that improves accuracy and reduces societal bias.

Findings

Existing tools are ~50% accurate and propagate societal biases.

Fine-tuned BERT achieves ~77% overall accuracy and 90% for non-binary classification.

ChatGPT shows 58% accuracy with zero-shot prompts on bias detection.

Abstract

AI models have become extremely popular and accessible to the general public. However, they are continuously under the scanner due to their demonstrable biases toward various sections of the society like people of color and non-binary people. In this study, we audit three existing gender analyzers -- uClassify, Readable and HackerFactor, for biases against non-binary individuals. These tools are designed to predict only the cisgender binary labels, which leads to discrimination against non-binary members of the society. We curate two datasets -- Reddit comments (660k) and, Tumblr posts (2.05M) and our experimental evaluation shows that the tools are highly inaccurate with the overall accuracy being ~50% on all platforms. Predictions for non-binary comments on all platforms are mostly female, thus propagating the societal bias that non-binary individuals are effeminate. To address this, we fine-tune a BERT multi-label classifier on the two datasets in multiple combinations, observe an overall performance of ~77% on the most realistically deployable setting and a surprisingly higher performance of 90% for the non-binary class. We also audit ChatGPT using zero-shot prompts on a small dataset (due to high pricing) and observe an average accuracy of 58% for Reddit and Tumblr combined (with overall better results for Reddit). Thus, we show that existing systems, including highly advanced ones like ChatGPT are biased, and need better audits and moderation and, that such societal biases can be addressed and alleviated through simple off-the-shelf models like BERT trained on more gender inclusive datasets.