Embedding Syntax and Semantics of Prepositions via Tensor Decomposition

TL;DR

This paper introduces a novel tensor decomposition method to embed prepositions by capturing their syntactic and semantic interactions, improving performance on attachment and selection tasks.

Contribution

It presents a new approach to preposition embedding using tensor decomposition of word-triple counts, revealing a unique geometric structure and enhancing downstream NLP tasks.

Findings

Achieves state-of-the-art results on preposition disambiguation datasets.

Demonstrates the utility of tensor-based embeddings in paraphrasing phrasal verbs.

Reveals a new geometric structure involving Hadamard products in preposition embeddings.

Abstract

Prepositions are among the most frequent words in English and play complex roles in the syntax and semantics of sentences. Not surprisingly, they pose well-known difficulties in automatic processing of sentences (prepositional attachment ambiguities and idiosyncratic uses in phrases). Existing methods on preposition representation treat prepositions no different from content words (e.g., word2vec and GloVe). In addition, recent studies aiming at solving prepositional attachment and preposition selection problems depend heavily on external linguistic resources and use dataset-specific word representations. In this paper we use word-triple counts (one of the triples being a preposition) to capture a preposition's interaction with its attachment and complement. We then derive preposition embeddings via tensor decomposition on a large unlabeled corpus. We reveal a new geometry involving Hadamard products and empirically demonstrate its utility in paraphrasing phrasal verbs. Furthermore, our preposition embeddings are used as simple features in two challenging downstream tasks: preposition selection and prepositional attachment disambiguation. We achieve results comparable to or better than the state-of-the-art on multiple standardized datasets.