GePBench: Evaluating Fundamental Geometric Perception for Multimodal Large Language Models

TL;DR

This paper introduces GePBench, a new benchmark to evaluate the geometric perception abilities of multimodal large language models, revealing current deficiencies and demonstrating improvements with targeted training.

Contribution

The paper presents GePBench, the first comprehensive benchmark for geometric perception in MLLMs, and shows how training with this data enhances model performance.

Findings

Current MLLMs perform poorly on geometric perception tasks.

Training with GePBench data significantly improves geometric understanding.

Geometric perception is crucial for advanced multimodal applications.

Abstract

Multimodal large language models (MLLMs) have made significant progress in integrating visual and linguistic understanding. Existing benchmarks typically focus on high-level semantic capabilities, such as scene understanding and visual reasoning, but often overlook a crucial, foundational ability: geometric perception. Geometric perception involves understanding geometric shapes, structures, and spatial relationships, which are essential for supporting higher-level semantic tasks. Despite its importance, this capability remains underexplored in current MLLM research. To address this gap, we introduce GePBench, a novel benchmark designed to assess the geometric perception abilities of MLLMs. Our extensive evaluations reveal that current state-of-the-art MLLMs exhibit significant deficiencies in geometric perception tasks. Furthermore, we show that models trained with GePBench data demonstrate substantial improvements on a wide range of benchmark tasks, highlighting the critical role of geometric perception in enabling advanced multimodal applications. Our code and datasets will be publicly available.