On the Effects of Knowledge-Augmented Data in Word Embeddings

TL;DR

This paper explores a new method for injecting linguistic knowledge into word embeddings via data augmentation, enhancing their semantic quality without compromising downstream task performance.

Contribution

It introduces a novel knowledge injection technique through data augmentation and systematically evaluates its impact on embedding quality and downstream tasks.

Findings

Improved intrinsic semantic properties of embeddings

No significant change in downstream classification performance

Effective knowledge augmentation method for word embeddings

Abstract

This paper investigates techniques for knowledge injection into word embeddings learned from large corpora of unannotated data. These representations are trained with word cooccurrence statistics and do not commonly exploit syntactic and semantic information from linguistic knowledge bases, which potentially limits their transferability to domains with differing language distributions or usages. We propose a novel approach for linguistic knowledge injection through data augmentation to learn word embeddings that enforce semantic relationships from the data, and systematically evaluate the impact it has on the resulting representations. We show our knowledge augmentation approach improves the intrinsic characteristics of the learned embeddings while not significantly altering their results on a downstream text classification task.