CopyNext: Explicit Span Copying and Alignment in Sequence to Sequence Models

TL;DR

CopyNext introduces an explicit span copying mechanism in seq2seq models, enabling precise token-level and span-level copying with explicit alignments, improving tasks like information extraction and achieving high accuracy and speed.

Contribution

It presents a novel explicit span copying and alignment method in seq2seq models, enhancing interpretability and efficiency for applications like NER.

Findings

Achieved near state-of-the-art accuracy on Nested NER.

Significantly increased decoding speed.

Provided explicit token and span alignments.

Abstract

Copy mechanisms are employed in sequence to sequence models (seq2seq) to generate reproductions of words from the input to the output. These frameworks, operating at the lexical type level, fail to provide an explicit alignment that records where each token was copied from. Further, they require contiguous token sequences from the input (spans) to be copied individually. We present a model with an explicit token-level copy operation and extend it to copying entire spans. Our model provides hard alignments between spans in the input and output, allowing for nontraditional applications of seq2seq, like information extraction. We demonstrate the approach on Nested Named Entity Recognition, achieving near state-of-the-art accuracy with an order of magnitude increase in decoding speed.