Deconstructing Word Embeddings

TL;DR

This paper critically reviews existing word embedding models, highlights their limitations, and introduces a new theoretical model called Derridian Embedding to address these issues.

Contribution

It presents a deconstructive analysis of current embeddings and proposes Derridian Embedding as a novel theoretical framework.

Findings

Current models show instability and geometric inconsistencies.

Derridian Embedding offers improved theoretical coherence.

Qualitative evaluation demonstrates advantages over existing models.

Abstract

A review of Word Embedding Models through a deconstructive approach reveals their several shortcomings and inconsistencies. These include instability of the vector representations, a distorted analogical reasoning, geometric incompatibility with linguistic features, and the inconsistencies in the corpus data. A new theoretical embedding model, Derridian Embedding, is proposed in this paper. Contemporary embedding models are evaluated qualitatively in terms of how adequate they are in relation to the capabilities of a Derridian Embedding.