# Cued@wmt19:ewc&lms

## TL;DR

This paper presents a combination of elastic weight consolidation and advanced language models to improve machine translation performance, achieving significant gains on WMT19 test sets through fine-tuning and novel LM architectures.

## Contribution

It introduces the use of EWC with checkpoint averaging and novel Transformer-based language models for enhanced translation quality.

## Key findings

- Significant improvements on WMT19 test sets
- Effective use of EWC with checkpoint averaging
- Enhanced language models based on Transformer architecture

## Abstract

Two techniques provide the fabric of the Cambridge University Engineering Department's (CUED) entry to the WMT19 evaluation campaign: elastic weight consolidation (EWC) and different forms of language modelling (LMs). We report substantial gains by fine-tuning very strong baselines on former WMT test sets using a combination of checkpoint averaging and EWC. A sentence-level Transformer LM and a document-level LM based on a modified Transformer architecture yield further gains. As in previous years, we also extract $n$-gram probabilities from SMT lattices which can be seen as a source-conditioned $n$-gram LM.