Gender bias in (non)-contextual clinical word embeddings for stereotypical medical categories

TL;DR

This paper investigates gender bias in clinical word embeddings across three medical categories, revealing that both contextualized and non-contextualized embeddings exhibit bias, with potential implications for medical applications.

Contribution

It provides a comparative analysis of gender bias in clinical embeddings, highlighting biases in both contextualized and non-contextualized models and their inconsistency with medical literature.

Findings

BioWordVec shows higher gender bias than clinical-BERT.

Both embeddings are biased towards sensitive gender groups.

Biases in embeddings conflict with medical literature.

Abstract

Clinical word embeddings are extensively used in various Bio-NLP problems as a state-of-the-art feature vector representation. Although they are quite successful at the semantic representation of words, due to the dataset - which potentially carries statistical and societal bias - on which they are trained, they might exhibit gender stereotypes. This study analyses gender bias of clinical embeddings on three medical categories: mental disorders, sexually transmitted diseases, and personality traits. To this extent, we analyze two different pre-trained embeddings namely (contextualized) clinical-BERT and (non-contextualized) BioWordVec. We show that both embeddings are biased towards sensitive gender groups but BioWordVec exhibits a higher bias than clinical-BERT for all three categories. Moreover, our analyses show that clinical embeddings carry a high degree of bias for some medical terms and diseases which is conflicting with medical literature. Having such an ill-founded relationship might cause harm in downstream applications that use clinical embeddings.