Training Kindai OCR with parallel textline images and self-attention feature distance-based loss

TL;DR

This paper proposes a novel training method for OCR on historical Japanese documents using parallel textline images and a self-attention feature distance loss, improving accuracy and domain adaptation.

Contribution

It introduces a distance-based loss function utilizing parallel images and self-attention features to enhance OCR performance on scarce historical data.

Findings

Reduced character error rate by up to 3.94%

Improved self-attention feature discrimination

Effective domain adaptation for historical OCR

Abstract

Kindai documents, written in modern Japanese from the late 19th to early 20th century, hold significant historical value for researchers studying societal structures, daily life, and environmental conditions of that period. However, transcribing these documents remains a labor-intensive and time-consuming task, resulting in limited annotated data for training optical character recognition (OCR) systems. This research addresses this challenge of data scarcity by leveraging parallel textline images - pairs of original Kindai text and their counterparts in contemporary Japanese fonts - to augment training datasets. We introduce a distance-based objective function that minimizes the gap between self-attention features of the parallel image pairs. Specifically, we explore Euclidean distance and Maximum Mean Discrepancy (MMD) as domain adaptation metrics. Experimental results demonstrate that our method reduces the character error rate (CER) by 2.23% and 3.94% over a Transformer-based OCR baseline when using Euclidean distance and MMD, respectively. Furthermore, our approach improves the discriminative quality of self-attention representations, leading to more effective OCR performance for historical documents.