Human-in-the-Loop Refinement of Word Embeddings

TL;DR

This paper introduces an interactive, human-in-the-loop system for refining word embeddings to address biases and quality issues, enabling tailored, iterative improvements and better understanding of their impact on machine learning tasks.

Contribution

It proposes a novel interactive refitting approach that allows humans to identify and correct biases in word embeddings iteratively and locally.

Findings

Enables fine-grained, organization-specific bias correction.

Facilitates iterative refinement of embeddings with human oversight.

Improves understanding of embedding impacts on downstream tasks.

Abstract

Word embeddings are a fixed, distributional representation of the context of words in a corpus learned from word co-occurrences. Despite their proven utility in machine learning tasks, word embedding models may capture uneven semantic and syntactic representations, and can inadvertently reflect various kinds of bias present within corpora upon which they were trained. It has been demonstrated that post-processing of word embeddings to apply information found in lexical dictionaries can improve the semantic associations, thus improving their quality. Building on this idea, we propose a system that incorporates an adaptation of word embedding post-processing, which we call "interactive refitting", to address some of the most daunting qualitative problems found in word embeddings. Our approach allows a human to identify and address potential quality issues with word embeddings interactively. This has the advantage of negating the question of who decides what constitutes bias or what other quality issues may affect downstream tasks. It allows each organization or entity to address concerns they may have at a fine grained level and to do so in an iterative and interactive fashion. It also allows for better insight into what effect word embeddings, and refinements to word embeddings, have on machine learning pipelines.