PharmaShip: An Entity-Centric, Reading-Order-Supervised Benchmark for Chinese Pharmaceutical Shipping Documents

TL;DR

PharmaShip introduces a comprehensive Chinese dataset for pharmaceutical shipping documents, emphasizing the importance of reading order and geometry-aware models to improve information extraction in noisy, real-world scenarios.

Contribution

The paper presents PharmaShip, a new benchmark dataset and evaluation protocol for entity recognition, relation extraction, and reading order prediction in Chinese pharmaceutical documents, highlighting the benefits of combining pixel and geometry information.

Findings

Explicit geometry and pixel information are complementary for model performance.

Reading-order regularization improves robustness and accuracy.

Longer positional coverage stabilizes predictions and reduces artifacts.

Abstract

We present PharmaShip, a real-world Chinese dataset of scanned pharmaceutical shipping documents designed to stress-test pre-trained text-layout models under noisy OCR and heterogeneous templates. PharmaShip covers three complementary tasks-sequence entity recognition (SER), relation extraction (RE), and reading order prediction (ROP)-and adopts an entity-centric evaluation protocol to minimize confounds across architectures. We benchmark five representative baselines spanning pixel-aware and geometry-aware families (LiLT, LayoutLMv3-base, GeoLayoutLM and their available RORE-enhanced variants), and standardize preprocessing, splits, and optimization. Experiments show that pixels and explicit geometry provide complementary inductive biases, yet neither alone is sufficient: injecting reading-order-oriented regularization consistently improves SER and EL and yields the most robust configuration, while longer positional coverage stabilizes late-page predictions and reduces truncation artifacts. ROP is accurate at the word level but challenging at the segment level, reflecting boundary ambiguity and long-range crossings. PharmaShip thus establishes a controlled, reproducible benchmark for safety-critical document understanding in the pharmaceutical domain and highlights sequence-aware constraints as a transferable bias for structure modeling. We release the dataset at https://github.com/KevinYuLei/PharmaShip.