SynJAC: Synthetic-data-driven Joint-granular Adaptation and Calibration for Domain Specific Scanned Document Key Information Extraction

TL;DR

SynJAC is a novel method that uses synthetic data and calibration techniques to improve key information extraction from scanned, visually rich documents with minimal manual labeling.

Contribution

It introduces a synthetic-data-driven approach combined with calibration for domain adaptation and key information extraction in scanned VRDs, reducing reliance on manual annotations.

Findings

Achieves competitive performance with limited manual labels.

Effectively adapts to domain-specific VRDs using synthetic data.

Demonstrates robustness across various scanned document types.

Abstract

Visually Rich Documents (VRDs), comprising elements such as charts, tables, and paragraphs, convey complex information across diverse domains. However, extracting key information from these documents remains labour-intensive, particularly for scanned formats with inconsistent layouts and domain-specific requirements. Despite advances in pretrained models for VRD understanding, their dependence on large annotated datasets for fine-tuning hinders scalability. This paper proposes \textbf{SynJAC} (Synthetic-data-driven Joint-granular Adaptation and Calibration), a method for key information extraction in scanned documents. SynJAC leverages synthetic, machine-generated data for domain adaptation and employs calibration on a small, manually annotated dataset to mitigate noise. By integrating fine-grained and coarse-grained document representation learning, SynJAC significantly reduces the need for extensive manual labelling while achieving competitive performance. Extensive experiments demonstrate its effectiveness in domain-specific and scanned VRD scenarios.