OCR-free Document Understanding Transformer

TL;DR

Donut is an OCR-free transformer-based model for document understanding that achieves state-of-the-art results, reducing computational costs and error propagation associated with traditional OCR-dependent methods.

Contribution

This paper introduces Donut, the first simple OCR-free transformer model for visual document understanding, with a novel pre-training approach and synthetic data generator for multilingual and multi-domain flexibility.

Findings

Achieves state-of-the-art performance on various VDU tasks.

Reduces computational costs compared to OCR-based methods.

Demonstrates robustness across languages and document types.

Abstract

Understanding document images (e.g., invoices) is a core but challenging task since it requires complex functions such as reading text and a holistic understanding of the document. Current Visual Document Understanding (VDU) methods outsource the task of reading text to off-the-shelf Optical Character Recognition (OCR) engines and focus on the understanding task with the OCR outputs. Although such OCR-based approaches have shown promising performance, they suffer from 1) high computational costs for using OCR; 2) inflexibility of OCR models on languages or types of document; 3) OCR error propagation to the subsequent process. To address these issues, in this paper, we introduce a novel OCR-free VDU model named Donut, which stands for Document understanding transformer. As the first step in OCR-free VDU research, we propose a simple architecture (i.e., Transformer) with a pre-training objective (i.e., cross-entropy loss). Donut is conceptually simple yet effective. Through extensive experiments and analyses, we show a simple OCR-free VDU model, Donut, achieves state-of-the-art performances on various VDU tasks in terms of both speed and accuracy. In addition, we offer a synthetic data generator that helps the model pre-training to be flexible in various languages and domains. The code, trained model and synthetic data are available at https://github.com/clovaai/donut.