Interactive Re-Fitting as a Technique for Improving Word Embeddings

TL;DR

This paper introduces an interactive method for refining word embeddings by allowing human adjustments to reduce bias and improve their semantic quality, building on prior post-processing techniques.

Contribution

It presents a novel interactive approach for post-processing word embeddings, enabling human-driven adjustments to improve quality and reduce bias.

Findings

Enables human interaction to refine embeddings

Allows targeted bias reduction in word spaces

Builds on existing post-processing methods

Abstract

Word embeddings are a fixed, distributional representation of the context of words in a corpus learned from word co-occurrences. While word embeddings have proven to have many practical uses in natural language processing tasks, they reflect the attributes of the corpus upon which they are trained. Recent work has demonstrated that post-processing of word embeddings to apply information found in lexical dictionaries can improve their quality. We build on this post-processing technique by making it interactive. Our approach makes it possible for humans to adjust portions of a word embedding space by moving sets of words closer to one another. One motivating use case for this capability is to enable users to identify and reduce the presence of bias in word embeddings. Our approach allows users to trigger selective post-processing as they interact with and assess potential bias in word embeddings.