A Question-Answering Approach to Key Value Pair Extraction from Form-like Document Images

TL;DR

This paper introduces KVPFormer, a novel question-answering based method utilizing Transformers for extracting key-value pairs from form-like document images, achieving state-of-the-art accuracy.

Contribution

The paper proposes a new QA-based approach with a coarse-to-fine prediction strategy and spatial attention bias for improved key-value extraction from document images.

Findings

Achieves state-of-the-art F1 scores on FUNSD and XFUND datasets.

Outperforms previous methods by 7.2% and 13.2% in F1 score.

Introduces spatial compatibility attention bias to enhance spatial interaction modeling.

Abstract

In this paper, we present a new question-answering (QA) based key-value pair extraction approach, called KVPFormer, to robustly extracting key-value relationships between entities from form-like document images. Specifically, KVPFormer first identifies key entities from all entities in an image with a Transformer encoder, then takes these key entities as \textbf{questions} and feeds them into a Transformer decoder to predict their corresponding \textbf{answers} (i.e., value entities) in parallel. To achieve higher answer prediction accuracy, we propose a coarse-to-fine answer prediction approach further, which first extracts multiple answer candidates for each identified question in the coarse stage and then selects the most likely one among these candidates in the fine stage. In this way, the learning difficulty of answer prediction can be effectively reduced so that the prediction accuracy can be improved. Moreover, we introduce a spatial compatibility attention bias into the self-attention/cross-attention mechanism for \Ours{} to better model the spatial interactions between entities. With these new techniques, our proposed \Ours{} achieves state-of-the-art results on FUNSD and XFUND datasets, outperforming the previous best-performing method by 7.2\% and 13.2\% in F1 score, respectively.