SPHINX: A Synthetic Environment for Visual Perception and Reasoning
Md Tanvirul Alam, Saksham Aggarwal, Justin Yang Chae, Nidhi Rastogi

TL;DR
Sphinx is a synthetic environment designed for evaluating and improving visual perception and reasoning in AI models, featuring diverse tasks and ground-truth solutions for large-scale dataset creation.
Contribution
The paper introduces Sphinx, a novel synthetic environment with verifiable puzzles for comprehensive visual reasoning evaluation and training.
Findings
State-of-the-art GPT-5 achieves only 51.1% accuracy on Sphinx tasks.
Reinforcement learning with verifiable rewards significantly improves model performance.
Sphinx enables large-scale dataset generation for diverse visual reasoning tasks.
Abstract
We present Sphinx, a synthetic environment for visual perception and reasoning that targets core cognitive primitives. Sphinx procedurally generates puzzles using motifs, tiles, charts, icons, and geometric primitives, each paired with verifiable ground-truth solutions, enabling both precise evaluation and large-scale dataset construction. The benchmark covers 25 task types spanning symmetry detection, geometric transformations, spatial reasoning, chart interpretation, and sequence prediction. Evaluating recent large vision-language models (LVLMs) shows that even state-of-the-art GPT-5 attains only 51.1% accuracy, well below human performance. Finally, we demonstrate that reinforcement learning with verifiable rewards (RLVR) substantially improves model accuracy on these tasks and yields gains on external visual reasoning benchmarks, highlighting its promise for advancing multimodal…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
