Doc2Dict: Information Extraction as Text Generation

TL;DR

Doc2Dict transforms information extraction into a text generation task using a transformer model trained on database records, simplifying workflows and handling long documents efficiently.

Contribution

It introduces a novel approach that replaces traditional IE pipelines with a direct text generation method trained on existing records, reducing annotation effort.

Findings

Competitive with complex pipelines

Handles documents up to 32,000 tokens

Simplifies IE workflow

Abstract

Typically, information extraction (IE) requires a pipeline approach: first, a sequence labeling model is trained on manually annotated documents to extract relevant spans; then, when a new document arrives, a model predicts spans which are then post-processed and standardized to convert the information into a database entry. We replace this labor-intensive workflow with a transformer language model trained on existing database records to directly generate structured JSON. Our solution removes the workload associated with producing token-level annotations and takes advantage of a data source which is generally quite plentiful (e.g. database records). As long documents are common in information extraction tasks, we use gradient checkpointing and chunked encoding to apply our method to sequences of up to 32,000 tokens on a single GPU. Our Doc2Dict approach is competitive with more complex, hand-engineered pipelines and offers a simple but effective baseline for document-level information extraction. We release our Doc2Dict model and code to reproduce our experiments and facilitate future work.