Fooling OCR Systems with Adversarial Text Images

TL;DR

This paper shows that advanced OCR systems can be fooled by slight, imperceptible modifications to printed text images, causing them to produce incorrect, semantically opposite transcriptions that can mislead downstream NLP applications.

Contribution

It introduces adversarial image techniques that can reliably alter OCR outputs without changing the visual appearance of the text.

Findings

Adversarial images can change OCR recognition results.

Minor modifications do not affect human reading.

OCR errors can be semantically opposite.

Abstract

We demonstrate that state-of-the-art optical character recognition (OCR) based on deep learning is vulnerable to adversarial images. Minor modifications to images of printed text, which do not change the meaning of the text to a human reader, cause the OCR system to "recognize" a different text where certain words chosen by the adversary are replaced by their semantic opposites. This completely changes the meaning of the output produced by the OCR system and by the NLP applications that use OCR for preprocessing their inputs.