An Empirical Study of Adequate Vision Span for Attention-Based Neural Machine Translation

TL;DR

This paper investigates the optimal size of the attention window in neural machine translation, proposing a dynamic framework that reduces redundant computations by over 50% with minimal accuracy loss.

Contribution

It introduces a novel attention framework that adaptively adjusts the vision span, significantly reducing computational complexity in translation models.

Findings

Over 50% reduction in average window size

Modest accuracy loss on English-Japanese and German-English tasks

Redundant computation in conventional attention mechanisms

Abstract

Recently, the attention mechanism plays a key role to achieve high performance for Neural Machine Translation models. However, as it computes a score function for the encoder states in all positions at each decoding step, the attention model greatly increases the computational complexity. In this paper, we investigate the adequate vision span of attention models in the context of machine translation, by proposing a novel attention framework that is capable of reducing redundant score computation dynamically. The term "vision span" means a window of the encoder states considered by the attention model in one step. In our experiments, we found that the average window size of vision span can be reduced by over 50% with modest loss in accuracy on English-Japanese and German-English translation tasks.% This results indicate that the conventional attention mechanism performs a significant amount of redundant computation.