Desiderata for Vector-Space Word Representations

TL;DR

This paper outlines essential desiderata for designing effective vector-space representations of words, emphasizing the importance of constraints and thoughtful design to enable their use in data processing tasks.

Contribution

It provides a comprehensive set of guidelines and criteria for creating meaningful and functional word vector representations, filling a gap in best practices.

Findings

Defines key desiderata for word vectors

Highlights importance of constraints in design

Guides future development of word representations

Abstract

A plethora of vector-space representations for words is currently available, which is growing. These consist of fixed-length vectors containing real values, which represent a word. The result is a representation upon which the power of many conventional information processing and data mining techniques can be brought to bear, as long as the representations are designed with some forethought and fit certain constraints. This paper details desiderata for the design of vector space representations of words.