Unsupervised Cross-Domain Word Representation Learning

TL;DR

This paper introduces an unsupervised approach for learning domain-specific word representations that capture semantic variations across different domains, improving performance in domain adaptation tasks.

Contribution

It proposes a novel unsupervised method using pivot words and an objective function to learn domain-specific embeddings, outperforming existing methods.

Findings

Significantly outperforms baseline models in domain adaptation tasks.

Achieves the best sentiment classification accuracies across multiple domain pairs.

Effectively captures domain-specific word semantics.

Abstract

Meaning of a word varies from one domain to another. Despite this important domain dependence in word semantics, existing word representation learning methods are bound to a single domain. Given a pair of \emph{source}-\emph{target} domains, we propose an unsupervised method for learning domain-specific word representations that accurately capture the domain-specific aspects of word semantics. First, we select a subset of frequent words that occur in both domains as \emph{pivots}. Next, we optimize an objective function that enforces two constraints: (a) for both source and target domain documents, pivots that appear in a document must accurately predict the co-occurring non-pivots, and (b) word representations learnt for pivots must be similar in the two domains. Moreover, we propose a method to perform domain adaptation using the learnt word representations. Our proposed method significantly outperforms competitive baselines including the state-of-the-art domain-insensitive word representations, and reports best sentiment classification accuracies for all domain-pairs in a benchmark dataset.