GroupLink: An End-to-end Multitask Method for Word Grouping and Relation Extraction in Form Understanding

TL;DR

This paper introduces GroupLink, an end-to-end multitask model that jointly performs word grouping and relation extraction in form understanding, leveraging multimodal features to improve accuracy on noisy scanned documents.

Contribution

The paper proposes a novel multitask approach that simultaneously handles word grouping and relation extraction, utilizing multimodal features for improved form understanding.

Findings

Effective on noisy scanned forms

Outperforms existing methods on FUNSD benchmark

Joint training enhances both tasks' accuracy

Abstract

Forms are a common type of document in real life and carry rich information through textual contents and the organizational structure. To realize automatic processing of forms, word grouping and relation extraction are two fundamental and crucial steps after preliminary processing of optical character reader (OCR). Word grouping is to aggregate words that belong to the same semantic entity, and relation extraction is to predict the links between semantic entities. Existing works treat them as two individual tasks, but these two tasks are correlated and can reinforce each other. The grouping process will refine the integrated representation of the corresponding entity, and the linking process will give feedback to the grouping performance. For this purpose, we acquire multimodal features from both textual data and layout information and build an end-to-end model through multitask training to combine word grouping and relation extraction to enhance performance on each task. We validate our proposed method on a real-world, fully-annotated, noisy-scanned benchmark, FUNSD, and extensive experiments demonstrate the effectiveness of our method.