PPTBench: Towards Holistic Evaluation of Large Language Models for PowerPoint Layout and Design Understanding

TL;DR

PPTBench is a comprehensive benchmark designed to evaluate large language models' ability to understand and generate PowerPoint slide layouts, revealing significant gaps in visual-structural reasoning and coherence.

Contribution

This work introduces PPTBench, a new multimodal benchmark with diverse tasks and data, to assess LLMs' performance on PowerPoint layout and design understanding, addressing limitations of prior narrow-focused benchmarks.

Findings

Models interpret slide content but struggle with spatial arrangements.

Current MLLMs have difficulty integrating visual cues with layout structures.

Systematic errors like misalignment and overlap are common in generated slides.

Abstract

PowerPoint presentations combine rich textual content with structured visual layouts, making them a natural testbed for evaluating the multimodal reasoning and layout understanding abilities of modern MLLMs. However, existing benchmarks focus solely on narrow subtasks while overlooking layout-centric challenges, which are central to real-world slide creation and editing. To bridge this gap, we introduce PPTBench, a comprehensive multimodal benchmark for evaluating LLMs on PowerPoint-related tasks. Leveraging a diverse source of 958 PPTX files, PPTBench evaluates models across four categories with 4,439 samples, including Detection, Understanding, Modification, and Generation. Our experiments reveal a substantial gap between semantic understanding and visual-layout reasoning in current MLLMs: models can interpret slide content but fail to produce coherent spatial arrangements. Ablation and further analysis show that current MLLMs struggle to combine visual cues with JSON-based layout structures and fail to integrate visual information into their API planning ability. And case studies visually expose systematic layout errors such as misalignment and element overlap. These findings provides a new perspective on evaluating VLLMs in PPT scenarios, highlighting challenges and directions for future research on visual-structural reasoning and coherent slide generation. All datasets and code are fully released to support reproducibility and future research.