WOVe: Incorporating Word Order in GloVe Word Embeddings
Mohammed Ibrahim, Susan Gauch, Tyler Gerth, Brandon Cox

TL;DR
This paper introduces WOVe, a method to incorporate word order into GloVe embeddings, significantly enhancing performance on analogy and similarity tasks.
Contribution
It proposes multiple approaches to embed word order into GloVe, improving its effectiveness in natural language understanding tasks.
Findings
WOVe outperforms GloVe on analogy completion by 36.34%
WOVe with concatenation increases word similarity accuracy by 2%
The methods significantly improve GloVe's performance on key NLP tasks
Abstract
Word vector representations open up new opportunities to extract useful information from unstructured text. Defining a word as a vector made it easy for the machine learning algorithms to understand a text and extract information from. Word vector representations have been used in many applications such word synonyms, word analogy, syntactic parsing, and many others. GloVe, based on word contexts and matrix vectorization, is an ef-fective vector-learning algorithm. It improves on previous vector-learning algorithms. However, the GloVe model fails to explicitly consider the order in which words appear within their contexts. In this paper, multiple methods of incorporating word order in GloVe word embeddings are proposed. Experimental results show that our Word Order Vector (WOVe) word embeddings approach outperforms unmodified GloVe on the natural lan-guage tasks of analogy completion…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Text and Document Classification Technologies
MethodsGloVe Embeddings
