From Plausibility to Verifiability: Risk-Controlled Generative OCR with Vision-Language Models

TL;DR

This paper addresses the deployment risks of generative OCR using vision-language models by introducing a geometric risk controller that improves reliability through structural screening and consensus checks.

Contribution

It proposes a model-agnostic Geometric Risk Controller that reduces extreme errors in generative OCR by applying structural verification and consensus mechanisms.

Findings

Significant reduction in extreme-error risk and over-generation.

Consistent improvements across multiple VLM backbones and OCR benchmarks.

Maintains predictable coverage while enhancing reliability.

Abstract

Modern vision-language models (VLMs) can act as generative OCR engines, yet open-ended decoding can expose rare but consequential failures. We identify a core deployment misalignment in generative OCR. Autoregressive decoding favors semantic plausibility, whereas OCR requires outputs that are visually grounded and geometrically verifiable. This mismatch produces severe errors, especially over-generation and unsupported substitutions, creating deployment risk even when benchmark accuracy remains high. We therefore formulate frozen VLM OCR as a selective accept/abstain problem and propose a model-agnostic Geometric Risk Controller. The controller probes multiple structured views of the same input, applies lightweight structural screening, and accepts a transcription only when cross-view consensus and stability satisfy predefined criteria, yielding a small family of operating points. Experiments on frozen VLM backbones and standard OCR benchmarks show consistent reductions in extreme-error risk and catastrophic over-generation at predictable coverage costs. Reliable deployment of generative OCR with frozen VLMs benefits from explicit system-level risk control rather than unconstrained generation.