SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gap

TL;DR

SCOB is a novel pre-training method that uses character-wise supervised contrastive learning with online text rendering to improve universal text understanding across diverse document and scene text domains, reducing annotation costs and bridging domain gaps.

Contribution

The paper introduces SCOB, a new pre-training approach that enhances cross-domain text understanding by leveraging contrastive learning and online text rendering, enabling weakly supervised learning.

Findings

SCOB improves performance over vanilla pre-training methods.

SCOB achieves comparable results to state-of-the-art techniques.

SCOB effectively bridges domain gaps in text understanding.

Abstract

Inspired by the great success of language model (LM)-based pre-training, recent studies in visual document understanding have explored LM-based pre-training methods for modeling text within document images. Among them, pre-training that reads all text from an image has shown promise, but often exhibits instability and even fails when applied to broader domains, such as those involving both visual documents and scene text images. This is a substantial limitation for real-world scenarios, where the processing of text image inputs in diverse domains is essential. In this paper, we investigate effective pre-training tasks in the broader domains and also propose a novel pre-training method called SCOB that leverages character-wise supervised contrastive learning with online text rendering to effectively pre-train document and scene text domains by bridging the domain gap. Moreover, SCOB enables weakly supervised learning, significantly reducing annotation costs. Extensive benchmarks demonstrate that SCOB generally improves vanilla pre-training methods and achieves comparable performance to state-of-the-art methods. Our findings suggest that SCOB can be served generally and effectively for read-type pre-training methods. The code will be available at https://github.com/naver-ai/scob.