Paraphrase-Supervised Models of Compositionality

TL;DR

This paper introduces a paraphrase-supervised approach for training compositional models of meaning, using automatic paraphrase extraction and context-aware scoring, leading to improved translation quality.

Contribution

It replaces manual annotations with automatic paraphrase data and develops a context-aware model for scoring compositionality, advancing semantic modeling techniques.

Findings

Paraphrase supervision matches previous methods in intrinsic evaluation.

Automatic methods reduce manual annotation effort.

Improved translation quality demonstrates practical benefits.

Abstract

Compositional vector space models of meaning promise new solutions to stubborn language understanding problems. This paper makes two contributions toward this end: (i) it uses automatically-extracted paraphrase examples as a source of supervision for training compositional models, replacing previous work which relied on manual annotations used for the same purpose, and (ii) develops a context-aware model for scoring phrasal compositionality. Experimental results indicate that these multiple sources of information can be used to learn partial semantic supervision that matches previous techniques in intrinsic evaluation tasks. Our approaches are also evaluated for their impact on a machine translation system where we show improvements in translation quality, demonstrating that compositionality in interpretation correlates with compositionality in translation.