DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information Extraction

TL;DR

DeepCPCFG combines deep learning with probabilistic context-free grammars to extract structured information from scanned documents without detailed annotations, achieving state-of-the-art results in invoice data extraction.

Contribution

It introduces DeepCPCFG, an end-to-end system that uses recursive neural networks and probabilistic grammars for document parsing without manual annotations.

Findings

Achieved state-of-the-art results on invoice data extraction

Successfully trained end-to-end with only relational-record labels

Avoided manual annotation costs

Abstract

We address the challenge of extracting structured information from business documents without detailed annotations. We propose Deep Conditional Probabilistic Context Free Grammars (DeepCPCFG) to parse two-dimensional complex documents and use Recursive Neural Networks to create an end-to-end system for finding the most probable parse that represents the structured information to be extracted. This system is trained end-to-end with scanned documents as input and only relational-records as labels. The relational-records are extracted from existing databases avoiding the cost of annotating documents by hand. We apply this approach to extract information from scanned invoices achieving state-of-the-art results despite using no hand-annotations.