Intrinsic analysis for dual word embedding space models

TL;DR

This study systematically compares dual embedding space models, specifically Word2Vec and GloVe, across various hyper-parameter settings to determine which configurations perform best on semantic, association, and analogy tasks.

Contribution

It provides a comprehensive evaluation of multiple hyper-parameter variations for dual embedding models, highlighting the effectiveness of non-default configurations across tasks.

Findings

GloVe non-default models outperform in all tasks.

Word2Vec non-default models outperform in 2 out of 3 tasks.

Extensive comparison across 84 models and datasets.

Abstract

Recent word embeddings techniques represent words in a continuous vector space, moving away from the atomic and sparse representations of the past. Each such technique can further create multiple varieties of embeddings based on different settings of hyper-parameters like embedding dimension size, context window size and training method. One additional variety appears when we especially consider the Dual embedding space techniques which generate not one but two-word embeddings as output. This gives rise to an interesting question - "is there one or a combination of the two word embeddings variety, which works better for a specific task?". This paper tries to answer this question by considering all of these variations. Herein, we compare two classical embedding methods belonging to two different methodologies - Word2Vec from window-based and Glove from count-based. For an extensive evaluation after considering all variations, a total of 84 different models were compared against semantic, association and analogy evaluations tasks which are made up of 9 open-source linguistics datasets. The final Word2vec reports showcase the preference of non-default model for 2 out of 3 tasks. In case of Glove, non-default models outperform in all 3 evaluation tasks.