LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding

TL;DR

LayoutXLM is a novel multimodal pre-trained model designed for multilingual visually-rich document understanding, significantly improving cross-lingual performance and introducing a new benchmark dataset XFUND for form understanding in seven languages.

Contribution

The paper introduces LayoutXLM, a new multimodal pre-trained model, and the XFUND dataset, enabling better multilingual document understanding and benchmarking across diverse languages.

Findings

LayoutXLM outperforms existing models on XFUND dataset.

The XFUND dataset covers 7 languages with manually labeled key-value pairs.

Pre-trained LayoutXLM and XFUND are publicly available.

Abstract

Multimodal pre-training with text, layout, and image has achieved SOTA performance for visually-rich document understanding tasks recently, which demonstrates the great potential for joint learning across different modalities. In this paper, we present LayoutXLM, a multimodal pre-trained model for multilingual document understanding, which aims to bridge the language barriers for visually-rich document understanding. To accurately evaluate LayoutXLM, we also introduce a multilingual form understanding benchmark dataset named XFUND, which includes form understanding samples in 7 languages (Chinese, Japanese, Spanish, French, Italian, German, Portuguese), and key-value pairs are manually labeled for each language. Experiment results show that the LayoutXLM model has significantly outperformed the existing SOTA cross-lingual pre-trained models on the XFUND dataset. The pre-trained LayoutXLM model and the XFUND dataset are publicly available at https://aka.ms/layoutxlm.