Robust Open-Vocabulary Translation from Visual Text Representations

TL;DR

This paper introduces a novel approach to machine translation using visual text representations, which enhances robustness to noise and variation compared to traditional subword-based models.

Contribution

It proposes visual text embeddings created from rendered text, improving robustness and matching traditional models' performance across datasets.

Findings

Visual text models match or outperform traditional models on standard datasets.

Visual embeddings significantly improve robustness to noise, e.g., in character permutation tasks.

Models with visual representations maintain high BLEU scores despite noise.

Abstract

Machine translation models have discrete vocabularies and commonly use subword segmentation techniques to achieve an 'open vocabulary.' This approach relies on consistent and correct underlying unicode sequences, and makes models susceptible to degradation from common types of noise and variation. Motivated by the robustness of human language processing, we propose the use of visual text representations, which dispense with a finite set of text embeddings in favor of continuous vocabularies created by processing visually rendered text with sliding windows. We show that models using visual text representations approach or match performance of traditional text models on small and larger datasets. More importantly, models with visual embeddings demonstrate significant robustness to varied types of noise, achieving e.g., 25.9 BLEU on a character permuted German-English task where subword models degrade to 1.9.