Neural semi-Markov CRF for Monolingual Word Alignment

TL;DR

This paper introduces a neural semi-Markov CRF model for monolingual word alignment that effectively captures phrase spans, outperforming previous methods and demonstrating utility in text simplification and classification.

Contribution

The paper proposes a novel neural semi-Markov CRF model that unifies word and phrase alignments using variable-length spans, and provides a new benchmark with human annotations for realistic evaluation.

Findings

Outperforms previous monolingual alignment methods.

Demonstrates strong generalizability across domains.

Improves downstream tasks like text simplification.

Abstract

Monolingual word alignment is important for studying fine-grained editing operations (i.e., deletion, addition, and substitution) in text-to-text generation tasks, such as paraphrase generation, text simplification, neutralizing biased language, etc. In this paper, we present a novel neural semi-Markov CRF alignment model, which unifies word and phrase alignments through variable-length spans. We also create a new benchmark with human annotations that cover four different text genres to evaluate monolingual word alignment models in more realistic settings. Experimental results show that our proposed model outperforms all previous approaches for monolingual word alignment as well as a competitive QA-based baseline, which was previously only applied to bilingual data. Our model demonstrates good generalizability to three out-of-domain datasets and shows great utility in two downstream applications: automatic text simplification and sentence pair classification tasks.