GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models

TL;DR

GGBench is a new benchmark designed to evaluate the geometric generative reasoning abilities of unified multimodal models, emphasizing their capacity for integrated understanding and active construction in visual and language tasks.

Contribution

This paper introduces GGBench, a novel benchmark that specifically measures the geometric reasoning and generative capabilities of multimodal models, filling a critical evaluation gap.

Findings

GGBench effectively diagnoses models' reasoning and construction skills.

Unified multimodal models show varied performance on geometric tasks.

The benchmark sets a new standard for evaluating generative reasoning in AI.

Abstract

The advent of Unified Multimodal Models (UMMs) signals a paradigm shift in artificial intelligence, moving from passive perception to active, cross-modal generation. Despite their unprecedented ability to synthesize information, a critical gap persists in evaluation: existing benchmarks primarily assess discriminative understanding or unconstrained image generation separately, failing to measure the integrated cognitive process of generative reasoning. To bridge this gap, we propose that geometric construction provides an ideal testbed as it inherently demands a fusion of language comprehension and precise visual generation. We introduce GGBench, a benchmark designed specifically to evaluate geometric generative reasoning. It provides a comprehensive framework for systematically diagnosing a model's ability to not only understand and reason but to actively construct a solution, thereby setting a more rigorous standard for the next generation of intelligent systems. Project website: https://opendatalab-raiser.github.io/GGBench/.