PGDP5K: A Diagram Parsing Dataset for Plane Geometry Problems

TL;DR

The paper introduces PGDP5K, a large-scale, finely annotated diagram dataset for plane geometry problems, aiming to advance diagram parsing and geometric reasoning research.

Contribution

It provides a new extensive dataset with detailed primitive-level annotations and a novel annotation method for plane geometry diagram parsing.

Findings

State-of-the-art methods achieve only 66.07% F1 on PGDP5K.

PGDP5K presents a challenging benchmark for future research.

The dataset enables automatic generation of geometric propositions.

Abstract

Diagram parsing is an important foundation for geometry problem solving, attracting increasing attention in the field of intelligent education and document image understanding. Due to the complex layout and between-primitive relationship, plane geometry diagram parsing (PGDP) is still a challenging task deserving further research and exploration. An appropriate dataset is critical for the research of PGDP. Although some datasets with rough annotations have been proposed to solve geometric problems, they are either small in scale or not publicly available. The rough annotations also make them not very useful. Thus, we propose a new large-scale geometry diagram dataset named PGDP5K and a novel annotation method. Our dataset consists of 5000 diagram samples composed of 16 shapes, covering 5 positional relations, 22 symbol types and 6 text types. Different from previous datasets, our PGDP5K dataset is labeled with more fine-grained annotations at primitive level, including primitive classes, locations and relationships. What is more, combined with above annotations and geometric prior knowledge, it can generate intelligible geometric propositions automatically and uniquely. We performed experiments on PGDP5K and IMP-Geometry3K datasets reveal that the state-of-the-art (SOTA) method achieves only 66.07% F1 value. This shows that PGDP5K presents a challenge for future research. Our dataset is available at http://www.nlpr.ia.ac.cn/databases/CASIA-PGDP5K/.