Sequence to Logic with Copy and Cache

TL;DR

This paper introduces a caching mechanism for sequence-to-logic models that generalizes copying, improving accuracy and robustness in logical form generation from natural language.

Contribution

It proposes a novel caching mechanism that extends copying by considering all source words based on their relevance to decoding context.

Findings

Improved sequence and token-level accuracy on logical form tasks.

Enhanced robustness against cross-domain adversarial attacks.

Effective generalization of copying mechanism through caching.

Abstract

Generating logical form equivalents of human language is a fresh way to employ neural architectures where long short-term memory effectively captures dependencies in both encoder and decoder units. The logical form of the sequence usually preserves information from the natural language side in the form of similar tokens, and recently a copying mechanism has been proposed which increases the probability of outputting tokens from the source input through decoding. In this paper we propose a caching mechanism as a more general form of the copying mechanism which also weighs all the words from the source vocabulary according to their relation to the current decoding context. Our results confirm that the proposed method achieves improvements in sequence/token-level accuracy on sequence to logical form tasks. Further experiments on cross-domain adversarial attacks show substantial improvements when using the most influential examples of other domains for training.