An Energy-based Model for Word-level AutoCompletion in Computer-aided Translation

TL;DR

This paper introduces an energy-based model for word-level auto-completion in computer-aided translation, improving performance by better leveraging source sentence information compared to traditional classification models.

Contribution

It proposes a novel energy-based approach for WLAC that enhances context understanding and demonstrates effective strategies to address training and inference challenges.

Findings

Achieves about 6.07% improvement over previous state-of-the-art models.

Effectively leverages source sentence information for better auto-completion.

Strategies significantly contribute to model performance.

Abstract

Word-level AutoCompletion(WLAC) is a rewarding yet challenging task in Computer-aided Translation. Existing work addresses this task through a classification model based on a neural network that maps the hidden vector of the input context into its corresponding label (i.e., the candidate target word is treated as a label). Since the context hidden vector itself does not take the label into account and it is projected to the label through a linear classifier, the model can not sufficiently leverage valuable information from the source sentence as verified in our experiments, which eventually hinders its overall performance. To alleviate this issue, this work proposes an energy-based model for WLAC, which enables the context hidden vector to capture crucial information from the source sentence. Unfortunately, training and inference suffer from efficiency and effectiveness challenges, thereby we employ three simple yet effective strategies to put our model into practice. Experiments on four standard benchmarks demonstrate that our reranking-based approach achieves substantial improvements (about 6.07%) over the previous state-of-the-art model. Further analyses show that each strategy of our approach contributes to the final performance.