Identity-sensitive Word Embedding through Heterogeneous Networks

TL;DR

This paper introduces a novel method for learning identity-sensitive word embeddings by constructing a heterogeneous network of words and their identities, capturing different meanings based on context, and demonstrating improved performance on classification and similarity tasks.

Contribution

It proposes a new approach to generate word embeddings that distinguish multiple word meanings using a heterogeneous network and network embedding techniques, which is a significant advancement over traditional methods.

Findings

Outperforms existing methods on text classification tasks.

Effectively captures multiple word meanings based on context.

Improves word similarity computation accuracy.

Abstract

Most existing word embedding approaches do not distinguish the same words in different contexts, therefore ignoring their contextual meanings. As a result, the learned embeddings of these words are usually a mixture of multiple meanings. In this paper, we acknowledge multiple identities of the same word in different contexts and learn the \textbf{identity-sensitive} word embeddings. Based on an identity-labeled text corpora, a heterogeneous network of words and word identities is constructed to model different-levels of word co-occurrences. The heterogeneous network is further embedded into a low-dimensional space through a principled network embedding approach, through which we are able to obtain the embeddings of words and the embeddings of word identities. We study three different types of word identities including topics, sentiments and categories. Experimental results on real-world data sets show that the identity-sensitive word embeddings learned by our approach indeed capture different meanings of words and outperforms competitive methods on tasks including text classification and word similarity computation.