End-to-End Hierarchical Relation Extraction for Generic Form Understanding

TL;DR

This paper introduces a novel end-to-end deep neural network that jointly performs entity detection and hierarchical relation extraction for form understanding, significantly improving accuracy on noisy scanned documents.

Contribution

The proposed model extends Multi-stage Attentional U-Net with Part-Intensity and Part-Association Fields for joint entity and link prediction, advancing form understanding methods.

Findings

Outperforms previous models on FUNSD dataset

Achieves higher accuracy in entity labeling

Improves entity linking performance

Abstract

Form understanding is a challenging problem which aims to recognize semantic entities from the input document and their hierarchical relations. Previous approaches face significant difficulty dealing with the complexity of the task, thus treat these objectives separately. To this end, we present a novel deep neural network to jointly perform both entity detection and link prediction in an end-to-end fashion. Our model extends the Multi-stage Attentional U-Net architecture with the Part-Intensity Fields and Part-Association Fields for link prediction, enriching the spatial information flow with the additional supervision from entity linking. We demonstrate the effectiveness of the model on the Form Understanding in Noisy Scanned Documents (FUNSD) dataset, where our method substantially outperforms the original model and state-of-the-art baselines in both Entity Labeling and Entity Linking task.