DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing

TL;DR

DECKBench is a comprehensive evaluation framework designed to benchmark multi-agent systems for generating and editing academic slide decks, addressing gaps in existing assessment protocols by focusing on fidelity, coherence, layout, and instruction following.

Contribution

This work introduces DECKBench, a novel benchmark with a curated dataset and evaluation protocol for multi-agent academic slide generation and editing, enabling standardized and reproducible assessments.

Findings

Benchmark exposes strengths and failure modes of systems.

Experimental results provide actionable insights for system improvements.

Establishes a foundation for reproducible evaluation of slide generation systems.

Abstract

Automatically generating and iteratively editing academic slide decks requires more than document summarization. It demands faithful content selection, coherent slide organization, layout-aware rendering, and robust multi-turn instruction following. However, existing benchmarks and evaluation protocols do not adequately measure these challenges. To address this gap, we introduce the Deck Edits and Compliance Kit Benchmark (DECKBench), an evaluation framework for multi-agent slide generation and editing. DECKBench is built on a curated dataset of paper to slide pairs augmented with realistic, simulated editing instructions. Our evaluation protocol systematically assesses slide-level and deck-level fidelity, coherence, layout quality, and multi-turn instruction following. We further implement a modular multi-agent baseline system that decomposes the slide generation and editing task into paper parsing and summarization, slide planning, HTML creation, and iterative editing. Experimental results demonstrate that the proposed benchmark highlights strengths, exposes failure modes, and provides actionable insights for improving multi-agent slide generation and editing systems. Overall, this work establishes a standardized foundation for reproducible and comparable evaluation of academic presentation generation and editing. Code and data are publicly available at https://github.com/morgan-heisler/DeckBench .