Learning Phrase Embeddings from Paraphrases with GRUs

TL;DR

This paper introduces a pairwise-GRU framework that leverages large paraphrase databases to generate versatile, compositional phrase embeddings, achieving state-of-the-art results in phrase similarity tasks.

Contribution

It presents a novel pairwise-GRU method that effectively learns phrase embeddings from paraphrases without extensive annotations or syntactic structures.

Findings

Achieves state-of-the-art performance on phrase similarity benchmarks.

Effective in generating phrase representations for diverse phrases.

Framework is reusable across different phrase types.

Abstract

Learning phrase representations has been widely explored in many Natural Language Processing (NLP) tasks (e.g., Sentiment Analysis, Machine Translation) and has shown promising improvements. Previous studies either learn non-compositional phrase representations with general word embedding learning techniques or learn compositional phrase representations based on syntactic structures, which either require huge amounts of human annotations or cannot be easily generalized to all phrases. In this work, we propose to take advantage of large-scaled paraphrase database and present a pair-wise gated recurrent units (pairwise-GRU) framework to generate compositional phrase representations. Our framework can be re-used to generate representations for any phrases. Experimental results show that our framework achieves state-of-the-art results on several phrase similarity tasks.