Data Extraction from Charts via Single Deep Neural Network

TL;DR

This paper introduces a single deep neural network framework for automatic data extraction from various chart types, addressing object relations and model generalization challenges in computer vision.

Contribution

The proposed framework unifies detection, recognition, and matching modules in one model, enabling effective extraction from multiple chart types with minimal revisions.

Findings

Achieves 79.4% accuracy on simulated bar charts

Achieves 88.0% accuracy on simulated pie charts

Performance drops to 57.5% and 62.3% outside training domain

Abstract

Automatic data extraction from charts is challenging for two reasons: there exist many relations among objects in a chart, which is not a common consideration in general computer vision problems; and different types of charts may not be processed by the same model. To address these problems, we propose a framework of a single deep neural network, which consists of object detection, text recognition and object matching modules. The framework handles both bar and pie charts, and it may also be extended to other types of charts by slight revisions and by augmenting the training data. Our model performs successfully on 79.4% of test simulated bar charts and 88.0% of test simulated pie charts, while for charts outside of the training domain it degrades for 57.5% and 62.3%, respectively.