LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoning

TL;DR

LatentGeo introduces a novel framework that learns continuous latent visual representations to internalize auxiliary geometric constructions, enhancing multimodal geometric reasoning without relying on explicit rendering or external tools.

Contribution

It proposes LatentGeo, a three-stage curriculum and reinforcement learning method to internalize geometric constructions in latent space, improving reasoning accuracy in multimodal models.

Findings

Significant performance improvements on GeoAux and MathVerse benchmarks.

Effective internalization of auxiliary geometric constructions without pixel rendering.

Validation through extensive ablation studies.

Abstract

Despite recent advances in multimodal reasoning, representing auxiliary geometric constructions remains a fundamental challenge for multimodal large language models (MLLMs). Such constructions are absent from the original diagram and must be introduced before theorems apply. Existing approaches predominantly rely on explicit construction paradigms, including text-based geometric specification, visual-token interleaving during reasoning, and tool-augmented geometric execution. However, these methods either fail to faithfully represent complex spatial relationships, incur representation mismatch between discrete symbols and continuous geometric structures, or rely on external capabilities that hinder end-to-end optimization. To address these limitations, we propose LatentGeo, a framework that learns continuous latent visual representations to internalize auxiliary geometric constructions without pixel-level rendering or external executors. We design a three-stage curriculum that progressively aligns and internalizes these latent representations through auxiliary visual supervision, followed by LaGDPO, a latent-aware reinforcement learning procedure that stabilizes latent representations during policy optimization while improving end-task correctness. To systematically evaluate construction-centric representation quality, we introduce GeoAux, a new benchmark targeting visually dependent geometry problems, and conduct experiments on GeoAux and MathVerse. Results show that LatentGeo achieves substantial gains on geometric reasoning tasks, particularly those requiring auxiliary constructions. Extensive analyses and ablation studies further validate the effectiveness of each component in our framework.