Enhancing Context Modeling with a Query-Guided Capsule Network for Document-level Translation

TL;DR

This paper introduces a query-guided capsule network approach to improve context modeling in document-level neural machine translation, leading to more coherent and consistent translations across various domains.

Contribution

It proposes a novel capsule network-based method that better captures relationships and roles of context words, surpassing traditional hierarchical attention models.

Findings

Significant performance improvements over strong baselines

Effective clustering of context information into relevant perspectives

Enhanced translation coherence and consistency

Abstract

Context modeling is essential to generate coherent and consistent translation for Document-level Neural Machine Translations. The widely used method for document-level translation usually compresses the context information into a representation via hierarchical attention networks. However, this method neither considers the relationship between context words nor distinguishes the roles of context words. To address this problem, we propose a query-guided capsule networks to cluster context information into different perspectives from which the target translation may concern. Experiment results show that our method can significantly outperform strong baselines on multiple data sets of different domains.