Learning Gender-Neutral Word Embeddings

TL;DR

This paper introduces a novel training method to create gender-neutral word embeddings that retain essential information while removing gender bias, improving fairness in NLP applications.

Contribution

The paper proposes a new training procedure for learning gender-neutral embeddings, exemplified by the GN-GloVe model, which isolates gender information without losing embedding utility.

Findings

GN-GloVe effectively isolates gender information.

The method preserves embedding functionality.

Gender bias is reduced in the resulting embeddings.

Abstract

Word embedding models have become a fundamental component in a wide range of Natural Language Processing (NLP) applications. However, embeddings trained on human-generated corpora have been demonstrated to inherit strong gender stereotypes that reflect social constructs. To address this concern, in this paper, we propose a novel training procedure for learning gender-neutral word embeddings. Our approach aims to preserve gender information in certain dimensions of word vectors while compelling other dimensions to be free of gender influence. Based on the proposed method, we generate a Gender-Neutral variant of GloVe (GN-GloVe). Quantitative and qualitative experiments demonstrate that GN-GloVe successfully isolates gender information without sacrificing the functionality of the embedding model.