An extensible point-based method for data chart value detection

TL;DR

This paper introduces an extensible point-based approach using a point proposal network to accurately identify and extract data points from complex scientific charts, including bar and pie charts, with high accuracy and robustness.

Contribution

The authors develop a novel, extensible point proposal network method for semantic point detection in data charts, capable of handling multiple chart types and trained effectively on synthetic data.

Findings

Achieves 0.8705 F1 on complex bar charts

Reaches 0.9810 F1 on synthetic charts

Performs well with synthetic training data on real charts

Abstract

We present an extensible method for identifying semantic points to reverse engineer (i.e. extract the values of) data charts, particularly those in scientific articles. Our method uses a point proposal network (akin to region proposal networks for object detection) to directly predict the position of points of interest in a chart, and it is readily extensible to multiple chart types and chart elements. We focus on complex bar charts in the scientific literature, on which our model is able to detect salient points with an accuracy of 0.8705 F1 (@1.5-cell max deviation); it achieves 0.9810 F1 on synthetically-generated charts similar to those used in prior works. We also explore training exclusively on synthetic data with novel augmentations, reaching surprisingly competent performance in this way (0.6621 F1) on real charts with widely varying appearance, and we further demonstrate our unchanged method applied directly to synthetic pie charts (0.8343 F1). Datasets, trained models, and evaluation code are available at https://github.com/BNLNLP/PPN_model.