CHARTER: heatmap-based multi-type chart data extraction

TL;DR

This paper introduces CHARTER, a neural network-based system that accurately extracts structured data from various types of charts in documents, using heatmap predictions trained on synthetic data.

Contribution

The method employs heatmap-based detection for precise chart element identification, overcoming bounding-box limitations and enabling end-to-end chart data extraction.

Findings

High robustness and precision demonstrated on benchmarks

Effective detection of pie, line, and scatter plots

Eliminates need for real training data through synthetic data use

Abstract

The digital conversion of information stored in documents is a great source of knowledge. In contrast to the documents text, the conversion of the embedded documents graphics, such as charts and plots, has been much less explored. We present a method and a system for end-to-end conversion of document charts into machine readable tabular data format, which can be easily stored and analyzed in the digital domain. Our approach extracts and analyses charts along with their graphical elements and supporting structures such as legends, axes, titles, and captions. Our detection system is based on neural networks, trained solely on synthetic data, eliminating the limiting factor of data collection. As opposed to previous methods, which detect graphical elements using bounding-boxes, our networks feature auxiliary domain specific heatmaps prediction enabling the precise detection of pie charts, line and scatter plots which do not fit the rectangular bounding-box presumption. Qualitative and quantitative results show high robustness and precision, improving upon previous works on popular benchmarks