MindBench: A Comprehensive Benchmark for Mind Map Structure Recognition and Analysis
Lei Chen, Feng Yan, Yujie Zhong, Shaoxiang Chen, Zequn Jie, Lin Ma

TL;DR
MindBench is a new comprehensive benchmark designed to evaluate and advance the ability of models to understand and analyze complex structured documents like mind maps and flowcharts, beyond simple text extraction.
Contribution
The paper introduces MindBench, a detailed benchmark with diverse tasks, annotations, and evaluation metrics specifically for structured document understanding.
Findings
Current models show significant room for improvement in structured document analysis.
MindBench enables comprehensive evaluation of text recognition, spatial awareness, and relationship understanding.
Experimental results highlight the potential and challenges in structured document comprehension.
Abstract
Multimodal Large Language Models (MLLM) have made significant progress in the field of document analysis. Despite this, existing benchmarks typically focus only on extracting text and simple layout information, neglecting the complex interactions between elements in structured documents such as mind maps and flowcharts. To address this issue, we introduce the new benchmark named MindBench, which not only includes meticulously constructed bilingual authentic or synthetic images, detailed annotations, evaluation metrics and baseline models, but also specifically designs five types of structured understanding and parsing tasks. These tasks include full parsing, partial parsing, position-related parsing, structured Visual Question Answering (VQA), and position-related VQA, covering key areas such as text recognition, spatial awareness, relationship discernment, and structured parsing.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCognitive Computing and Networks · EEG and Brain-Computer Interfaces
MethodsFocus
