Synthetic Perception: Can Generated Images Unlock Latent Visual Prior for Text-Centric Reasoning?

TL;DR

This paper explores whether on-the-fly generated images by Text-to-Image models can enhance text-centric reasoning by bridging the modality gap, showing significant performance improvements under certain conditions.

Contribution

It systematically evaluates the use of synthetic images to unlock visual priors for language tasks, establishing a benchmark and analyzing key factors affecting effectiveness.

Findings

Synthetic perception improves text classification performance.

Effectiveness depends on image quality and semantic alignment.

Provides a new cross-modal probing paradigm for language understanding.

Abstract

A significant ``modality gap" exists between the abundance of text-only data and the increasing power of multimodal models. This work systematically investigates whether images generated on-the-fly by Text-to-Image (T2I) models can serve as a mechanism to unlock latent visual priors for text-centric reasoning. Through a comprehensive evaluation framework on text classification, we analyze the impact of critical variables, including T2I model quality (e.g., Flux.1, SDXL), prompt engineering strategies, and multimodal fusion architectures. Our findings demonstrate that this ``synthetic perception" can yield significant performance gains by effectively projecting text into a visual semantic space, even when augmenting strong large language model baselines like Llama-3 and Qwen-2.5. We show that this approach serves as a form of cross-modal probing, mitigating the sensory deprivation inherent in pure text training. However, the effectiveness is highly conditional, depending on the semantic alignment between text and the generated image, the task's visual groundability, and the generative fidelity of the T2I model. Our work establishes a rigorous benchmark for this paradigm, demonstrating its viability as a pathway to enrich language understanding in traditionally unimodal scenarios.