# Fair is Better than Sensational:Man is to Doctor as Woman is to Doctor

## TL;DR

This paper critiques the use of analogy tasks for detecting biases in word embeddings, highlighting their limitations and potential to distort understanding of human biases encoded in language models.

## Contribution

The paper clarifies the shortcomings of analogy-based bias detection and argues that analogies are not suitable tools for measuring bias in word embeddings.

## Key findings

- Analogies can misrepresent the presence of bias in embeddings.
- Bias in word embeddings should be addressed with more appropriate methods.
- Analogies may exaggerate or hide certain biases.

## Abstract

Analogies such as "man is to king as woman is to X" are often used to illustrate the amazing power of word embeddings. Concurrently, they have also been used to expose how strongly human biases are encoded in vector spaces built on natural language, like "man is to computer programmer as woman is to homemaker". Recent work has shown that analogies are in fact not such a diagnostic for bias, and other methods have been proven to be more apt to the task. However, beside the intrinsic problems with the analogy task as a bias detection tool, in this paper we show that a series of issues related to how analogies have been implemented and used might have yielded a distorted picture of bias in word embeddings. Human biases are present in word embeddings and need to be addressed. Analogies, though, are probably not the right tool to do so. Also, the way they have been most often used has exacerbated some possibly non-existing biases and perhaps hid others. Because they are still widely popular, and some of them have become classics within and outside the NLP community, we deem it important to provide a series of clarifications that should put well-known, and potentially new cases into the right perspective.