SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinking

TL;DR

This paper introduces SemVink, a simple low-resolution image scaling technique that dramatically improves VLMs' ability to detect hidden content in optical illusions, revealing their overreliance on high-level semantics.

Contribution

The paper presents SemVink, a novel low-resolution scaling method that enhances VLMs' perceptual capabilities and exposes architectural limitations in current models.

Findings

VLMs achieve near-zero accuracy on optical illusions without special prompting.

Scaling images to low resolutions (>99% accuracy) overcomes VLMs' perceptual limitations.

Highlights the need for hybrid models with multi-scale processing for robust visual understanding.

Abstract

Vision-language models (VLMs) excel in semantic tasks but falter at a core human capability: detecting hidden content in optical illusions or AI-generated images through perceptual adjustments like zooming. We introduce HC-Bench, a benchmark of 112 images with hidden text, objects, and illusions, revealing that leading VLMs achieve near-zero accuracy (0-5.36%)-even with explicit prompting. Humans resolve such ambiguities instinctively, yet VLMs fail due to an overreliance on high-level semantics. Strikingly, we propose SemVink (Semantic Visual Thinking) by simply scaling images to low resolutions (32-128 pixels), which unlocks >99% accuracy by eliminating redundant visual noise. This exposes a critical architectural flaw: VLMs prioritize abstract reasoning over low-level visual operations crucial for real-world robustness. Our work urges a shift toward hybrid models integrating multi-scale processing, bridging the gap between computational vision and human cognition for applications in medical imaging, security, and beyond.