# An Analysis of Source-Side Grammatical Errors in NMT

## TL;DR

This paper investigates how grammatical errors in source sentences affect neural machine translation quality, introducing new evaluation methods and visualization techniques to analyze NMT robustness against real-world grammatical noise.

## Contribution

It provides the first large-scale analysis of English-German NMT performance on grammatical errors and proposes novel evaluation and visualization methods for robustness analysis.

## Key findings

- Grammatical errors significantly degrade NMT output quality.
- New evaluation methods enable robustness assessment without true references.
- Visualization techniques offer insights into divergence caused by errors.

## Abstract

The quality of Neural Machine Translation (NMT) has been shown to significantly degrade when confronted with source-side noise. We present the first large-scale study of state-of-the-art English-to-German NMT on real grammatical noise, by evaluating on several Grammar Correction corpora. We present methods for evaluating NMT robustness without true references, and we use them for extensive analysis of the effects that different grammatical errors have on the NMT output. We also introduce a technique for visualizing the divergence distribution caused by a source-side error, which allows for additional insights.