A Deeper Look into Dependency-Based Word Embeddings

TL;DR

This paper examines how different dependency-based word embeddings affect various linguistic similarity tasks and downstream applications, highlighting the benefits of enhanced dependency contexts in improving embedding performance.

Contribution

It provides a comprehensive analysis of dependency-based embeddings trained with various dependency types and enhancements, comparing their effectiveness across multiple tasks.

Findings

Universal and Stanford dependency embeddings excel at different tasks.

Enhanced dependency contexts often improve embedding performance.

Dependency-based embeddings outperform basic linear context embeddings.

Abstract

We investigate the effect of various dependency-based word embeddings on distinguishing between functional and domain similarity, word similarity rankings, and two downstream tasks in English. Variations include word embeddings trained using context windows from Stanford and Universal dependencies at several levels of enhancement (ranging from unlabeled, to Enhanced++ dependencies). Results are compared to basic linear contexts and evaluated on several datasets. We found that embeddings trained with Universal and Stanford dependency contexts excel at different tasks, and that enhanced dependencies often improve performance.