Kyrtos: A methodology for automatic deep analysis of graphic charts with curves in technical documents

TL;DR

Kyrtos is a novel methodology that automatically recognizes, analyzes, and converts curve charts in technical documents into structured graphs and natural language descriptions, enhancing document understanding.

Contribution

It introduces a clustering-based recognition approach and a graph-based analysis for structural and behavioral features of chart curves, enabling detailed interpretation.

Findings

High accuracy in recognizing and analyzing chart curves

Effective conversion of curves into attributed graphs and natural language

Demonstrated robustness across multiple chart functions

Abstract

Deep Understanding of Technical Documents (DUTD) has become a very attractive field with great potential due to large amounts of accumulated documents and the valuable knowledge contained in them. In addition, the holistic understanding of technical documents depends on the accurate analysis of its particular modalities, such as graphics, tables, diagrams, text, etc. and their associations. In this paper, we introduce the Kyrtos methodology for the automatic recognition and analysis of charts with curves in graphics images of technical documents. The recognition processing part adopts a clustering based approach to recognize middle-points that delimit the line-segments that construct the illustrated curves. The analysis processing part parses the extracted line-segments of curves to capture behavioral features such as direction, trend and etc. These associations assist the conversion of recognized segments' relations into attributed graphs, for the preservation of the curves' structural characteristics. The graph relations are also are expressed into natural language (NL) text sentences, enriching the document's text and facilitating their conversion into Stochastic Petri-net (SPN) graphs, which depict the internal functionality represented in the chart image. Extensive evaluation results demonstrate the accuracy of Kyrtos' recognition and analysis methods by measuring the structural similarity between input chart curves and the approximations generated by Kyrtos for charts with multiple functions.