Learning Better Word Embedding by Asymmetric Low-Rank Projection of Knowledge Graph

TL;DR

This paper introduces ProjectNet, a novel method that enhances word embeddings by modeling complex relationships in knowledge graphs through asymmetric low-rank projections, improving NLP task performance.

Contribution

It proposes a new approach using asymmetric low-rank projections to better capture diverse relationships and semantic differences in knowledge graphs for improved word embeddings.

Findings

ProjectNet outperforms previous methods in word embedding quality.

Improved embeddings lead to better NLP task results.

Model effectively captures complex entity relationships.

Abstract

Word embedding, which refers to low-dimensional dense vector representations of natural words, has demonstrated its power in many natural language processing tasks. However, it may suffer from the inaccurate and incomplete information contained in the free text corpus as training data. To tackle this challenge, there have been quite a few works that leverage knowledge graphs as an additional information source to improve the quality of word embedding. Although these works have achieved certain success, they have neglected some important facts about knowledge graphs: (i) many relationships in knowledge graphs are \emph{many-to-one}, \emph{one-to-many} or even \emph{many-to-many}, rather than simply \emph{one-to-one}; (ii) most head entities and tail entities in knowledge graphs come from very different semantic spaces. To address these issues, in this paper, we propose a new algorithm named ProjectNet. ProjecNet models the relationships between head and tail entities after transforming them with different low-rank projection matrices. The low-rank projection can allow non \emph{one-to-one} relationships between entities, while different projection matrices for head and tail entities allow them to originate in different semantic spaces. The experimental results demonstrate that ProjectNet yields more accurate word embedding than previous works, thus leads to clear improvements in various natural language processing tasks.