PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition

TL;DR

PageNet introduces an end-to-end weakly supervised approach for page-level handwritten Chinese text recognition, effectively handling complex layouts with minimal annotation effort, and outperforms existing methods on multiple datasets.

Contribution

It presents a novel weakly supervised framework for page-level handwritten Chinese text recognition that detects, recognizes, and predicts reading order with only transcript annotations.

Findings

Outperforms existing weakly and fully supervised methods on five datasets.

Handles complex layouts including multi-directional and curved text lines.

Requires only transcript annotations, reducing labeling costs.

Abstract

Handwritten Chinese text recognition (HCTR) has been an active research topic for decades. However, most previous studies solely focus on the recognition of cropped text line images, ignoring the error caused by text line detection in real-world applications. Although some approaches aimed at page-level text recognition have been proposed in recent years, they either are limited to simple layouts or require very detailed annotations including expensive line-level and even character-level bounding boxes. To this end, we propose PageNet for end-to-end weakly supervised page-level HCTR. PageNet detects and recognizes characters and predicts the reading order between them, which is more robust and flexible when dealing with complex layouts including multi-directional and curved text lines. Utilizing the proposed weakly supervised learning framework, PageNet requires only transcripts to be annotated for real data; however, it can still output detection and recognition results at both the character and line levels, avoiding the labor and cost of labeling bounding boxes of characters and text lines. Extensive experiments conducted on five datasets demonstrate the superiority of PageNet over existing weakly supervised and fully supervised page-level methods. These experimental results may spark further research beyond the realms of existing methods based on connectionist temporal classification or attention. The source code is available at https://github.com/shannanyinxiang/PageNet.