When Text-as-Vision Meets Semantic IDs in Generative Recommendation: An Empirical Study

TL;DR

This paper explores using OCR-based visual representations of item descriptions for Semantic ID learning in generative recommendation, demonstrating improved robustness and effectiveness over traditional text encoders, especially in multimodal settings.

Contribution

The study systematically evaluates OCR-based visual text representations for Semantic ID learning, showing they outperform standard text encoders in various recommendation scenarios.

Findings

OCR-text matches or exceeds standard text embeddings in performance.

OCR-based Semantic IDs are robust under spatial-resolution compression.

OCR representations improve cross-modal fusion stability.

Abstract

Semantic ID learning is a key interface in Generative Recommendation (GR) models, mapping items to discrete identifiers grounded in side information, most commonly via a pretrained text encoder. However, these text encoders are primarily optimized for well-formed natural language. In real-world recommendation data, item descriptions are often symbolic and attribute-centric, containing numerals, units, and abbreviations. These text encoders can break these signals into fragmented tokens, weakening semantic coherence and distorting relationships among attributes. Worse still, when moving to multimodal GR, relying on standard text encoders introduces an additional obstacle: text and image embeddings often exhibit mismatched geometric structures, making cross-modal fusion less effective and less stable. In this paper, we revisit representation design for Semantic ID learning by treating text as a visual signal. We conduct a systematic empirical study of OCR-based text representations, obtained by rendering item descriptions into images and encoding them with vision-based OCR models. Experiments across four datasets and two generative backbones show that OCR-text consistently matches or surpasses standard text embeddings for Semantic ID learning in both unimodal and multimodal settings. Furthermore, we find that OCR-based Semantic IDs remain robust under extreme spatial-resolution compression, indicating strong robustness and efficiency in practical deployments.