SpaDen : Sparse and Dense Keypoint Estimation for Real-World Chart Understanding

TL;DR

This paper presents SpaDen, a novel bottom-up method that combines sparse and dense keypoint detection with deep metric learning to accurately extract and reconstruct data from real-world charts.

Contribution

It introduces a fusion of continuous and discrete keypoints with a self-attention feature-fusion layer and deep metric learning for unsupervised clustering in chart understanding.

Findings

Effective keypoint detection improves chart component reconstruction.

Combining sparse and dense objectives enhances data extraction accuracy.

Extensive experiments validate the method's robustness on real-world charts.

Abstract

We introduce a novel bottom-up approach for the extraction of chart data. Our model utilizes images of charts as inputs and learns to detect keypoints (KP), which are used to reconstruct the components within the plot area. Our novelty lies in detecting a fusion of continuous and discrete KP as predicted heatmaps. A combination of sparse and dense per-pixel objectives coupled with a uni-modal self-attention-based feature-fusion layer is applied to learn KP embeddings. Further leveraging deep metric learning for unsupervised clustering, allows us to segment the chart plot area into various objects. By further matching the chart components to the legend, we are able to obtain the data series names. A post-processing threshold is applied to the KP embeddings to refine the object reconstructions and improve accuracy. Our extensive experiments include an evaluation of different modules for KP estimation and the combination of deep layer aggregation and corner pooling approaches. The results of our experiments provide extensive evaluation for the task of real-world chart data extraction.