XFormParser: A Simple and Effective Multimodal Multilingual Semi-structured Form Parser

TL;DR

XFormParser is a novel multimodal and multilingual form parser that combines semantic entity recognition and relation extraction within a Transformer-based framework, significantly improving industrial form parsing accuracy and robustness.

Contribution

The paper introduces XFormParser, a new semi-structured form parser that integrates SER and RE, enhances multilingual parsing with Bi-LSTM, and provides a new industrial dataset InDFormSFT.

Findings

Achieves up to 1.79% F1 score improvement over SOTA in RE tasks.

Demonstrates strong performance in multilingual and zero-shot settings.

Proves robustness across various industrial form parsing benchmarks.

Abstract

In the domain of Document AI, parsing semi-structured image form is a crucial Key Information Extraction (KIE) task. The advent of pre-trained multimodal models significantly empowers Document AI frameworks to extract key information from form documents in different formats such as PDF, Word, and images. Nonetheless, form parsing is still encumbered by notable challenges like subpar capabilities in multilingual parsing and diminished recall in industrial contexts in rich text and rich visuals. In this work, we introduce a simple but effective \textbf{M}ultimodal and \textbf{M}ultilingual semi-structured \textbf{FORM} \textbf{PARSER} (\textbf{XFormParser}), which anchored on a comprehensive Transformer-based pre-trained language model and innovatively amalgamates semantic entity recognition (SER) and relation extraction (RE) into a unified framework. Combined with Bi-LSTM, the performance of multilingual parsing is significantly improved. Furthermore, we develop InDFormSFT, a pioneering supervised fine-tuning (SFT) industrial dataset that specifically addresses the parsing needs of forms in various industrial contexts. XFormParser has demonstrated its unparalleled effectiveness and robustness through rigorous testing on established benchmarks. Compared to existing state-of-the-art (SOTA) models, XFormParser notably achieves up to 1.79\% F1 score improvement on RE tasks in language-specific settings. It also exhibits exceptional cross-task performance improvements in multilingual and zero-shot settings. The codes, datasets, and pre-trained models are publicly available at https://github.com/zhbuaa0/xformparser.