The Expressive Power of Word Embeddings
Yanqing Chen, Bryan Perozzi, Rami Al-Rfou, Steven Skiena

TL;DR
This paper investigates the semantic capabilities of various word embeddings, revealing significant differences in their quality and the influence of their dimensionality on capturing nuanced meanings for NLP tasks.
Contribution
It introduces tasks to evaluate embeddings' semantic properties and analyzes how dimensionality affects their effectiveness, providing insights into embedding quality.
Findings
Embeddings capture nuanced semantics even without sentence structure.
Significant variance exists in the quality of different embeddings.
Dimensionality impacts the features and effectiveness of embeddings.
Abstract
We seek to better understand the difference in quality of the several publicly released embeddings. We propose several tasks that help to distinguish the characteristics of different embeddings. Our evaluation of sentiment polarity and synonym/antonym relations shows that embeddings are able to capture surprisingly nuanced semantics even in the absence of sentence structure. Moreover, benchmarking the embeddings shows great variance in quality and characteristics of the semantics captured by the tested embeddings. Finally, we show the impact of varying the number of dimensions and the resolution of each dimension on the effective useful features captured by the embedding space. Our contributions highlight the importance of embeddings for NLP tasks and the effect of their quality on the final results.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Sentiment Analysis and Opinion Mining
