On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation

TL;DR

This paper investigates how neural machine translation models encode sentence structure, revealing that superficial cues influence neuron activations and demonstrating partial control over syntactic output through neuron manipulation.

Contribution

The study introduces a model-agnostic methodology to analyze sentence structure encoding in neural translation models and develops a semi-automatic paraphrase generation technique.

Findings

Neuron activation similarity is mainly due to word choice and sentence length.

Neuron manipulation can influence syntactic form of translation output.

Deep models encode sentence-structure distinctions despite neuron-level evidence being limited.

Abstract

We present a methodology that explores how sentence structure is reflected in neural representations of machine translation systems. We demonstrate our model-agnostic approach with the Transformer English-German translation model. We analyze neuron-level correlation of activations between paraphrases while discussing the methodology challenges and the need for confound analysis to isolate the effects of shallow cues. We find that similarity between activation patterns can be mostly accounted for by similarity in word choice and sentence length. Following that, we manipulate neuron activations to control the syntactic form of the output. We show this intervention to be somewhat successful, indicating that deep models capture sentence-structure distinctions, despite finding no such indication at the neuron level. To conduct our experiments, we develop a semi-automatic method to generate meaning-preserving minimal pair paraphrases (active-passive voice and adverbial clause-noun phrase) and compile a corpus of such pairs.