Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR

TL;DR

This paper critically evaluates DeepSeek-OCR's reliance on linguistic priors versus visual merit, revealing its performance drops significantly without language support and highlighting limitations in current optical compression for long-context tasks.

Contribution

It provides an empirical analysis of DeepSeek-OCR's true OCR capabilities, distinguishing visual from linguistic contributions, and benchmarks its robustness against semantic perturbations.

Findings

DeepSeek-OCR's performance drops from 90% to 20% without linguistic support.

Traditional OCR methods are more robust to semantic corruption than end-to-end models.

Lower visual token counts increase reliance on language priors, risking hallucinations.

Abstract

DeepSeek-OCR utilizes an optical 2D mapping approach to achieve high-ratio vision-text compression, claiming to decode text tokens exceeding ten times the input visual tokens. While this suggests a promising solution for the LLM long-context bottleneck, we investigate a critical question: "Visual merit or linguistic crutch - which drives DeepSeek-OCR's performance?" By employing sentence-level and word-level semantic corruption, we isolate the model's intrinsic OCR capabilities from its language priors. Results demonstrate that without linguistic support, DeepSeek-OCR's performance plummets from approximately 90% to 20%. Comparative benchmarking against 13 baseline models reveals that traditional pipeline OCR methods exhibit significantly higher robustness to such semantic perturbations than end-to-end methods. Furthermore, we find that lower visual token counts correlate with increased reliance on priors, exacerbating hallucination risks. Context stress testing also reveals a total model collapse around 10,000 text tokens, suggesting that current optical compression techniques may paradoxically aggravate the long-context bottleneck. This study empirically defines DeepSeek-OCR's capability boundaries and offers essential insights for future optimizations of the vision-text compression paradigm. We release all data, results and scripts used in this study at https://github.com/dududuck00/DeepSeekOCR.