Understanding Cross-Language Transfer Improvements in Low-Resource HTR: The Role of Sequence Modeling

TL;DR

This study investigates how sequence modeling, rather than shared visual features, enhances cross-language transfer in low-resource handwritten text recognition for Arabic-script languages, emphasizing the importance of contextual dependencies.

Contribution

It provides a controlled comparison showing sequence-level modeling is key to transfer improvements, beyond shared visual representations, in low-resource HTR.

Findings

CRNN models outperform CNN-only models in multi-script transfer scenarios.

Sequence-level modeling correlates with improved transfer performance.

Shared visual features alone are insufficient for effective cross-language transfer.

Abstract

Handwritten Text Recognition (HTR) for Arabic-script languages benefits from cross-language joint training under low-resource conditions, particularly when using CRNN-based models that combine convolutional encoders with sequence modeling. However, it remains unclear whether these improvements are better explained by shared visual representations or sequence-level dependencies. In this work, we conduct a controlled architectural study of line-level Arabic-script HTR, comparing CNN-only models with CTC decoding and CRNN models under identical single-script and multi-script training regimes. Experiments are performed on Arabic (KHATT), Urdu (NUST-UHWR), and Persian (PHTD) datasets under low-resource settings (K in {100, 500, 1000}). Our results show a clear divergence in transfer behavior: while CNN-only models exhibit limited or unstable improvements, CRNN models achieve better performance under multi-script training, particularly in the most data-constrained regimes. Focusing on transfer improvements (delta CER) rather than absolute performance, we find that cross-language improvements are associated with sequence-level modeling, while sharing visual representations learned by the CNN encoder, corresponding to similarities in character shapes across scripts, alone appears to be insufficient. This finding suggests that contextual modeling plays an important role in enabling effective transfer in low-resource scenarios, and that similar behavior may extend to other low-resource language settings.