Image-based table recognition: data, model, and evaluation

TL;DR

This paper introduces PubTabNet, a large dataset for image-based table recognition, and proposes a novel attention-based model that significantly improves recognition accuracy of complex tables from images.

Contribution

The paper presents the largest publicly available dataset for table recognition and a new attention-based encoder-dual-decoder model with a novel TEDS metric for better evaluation.

Findings

The EDD model outperforms previous methods by 9.7% in TEDS score.

PubTabNet contains 568k table images with HTML annotations.

The TEDS metric better captures multi-hop cell misalignments and OCR errors.

Abstract

Important information that relates to a specific topic in a document is often organized in tabular format to assist readers with information retrieval and comparison, which may be difficult to provide in natural language. However, tabular data in unstructured digital documents, e.g., Portable Document Format (PDF) and images, are difficult to parse into structured machine-readable format, due to complexity and diversity in their structure and style. To facilitate image-based table recognition with deep learning, we develop the largest publicly available table recognition dataset PubTabNet (https://github.com/ibm-aur-nlp/PubTabNet), containing 568k table images with corresponding structured HTML representation. PubTabNet is automatically generated by matching the XML and PDF representations of the scientific articles in PubMed Central Open Access Subset (PMCOA). We also propose a novel attention-based encoder-dual-decoder (EDD) architecture that converts images of tables into HTML code. The model has a structure decoder which reconstructs the table structure and helps the cell decoder to recognize cell content. In addition, we propose a new Tree-Edit-Distance-based Similarity (TEDS) metric for table recognition, which more appropriately captures multi-hop cell misalignment and OCR errors than the pre-established metric. The experiments demonstrate that the EDD model can accurately recognize complex tables solely relying on the image representation, outperforming the state-of-the-art by 9.7% absolute TEDS score.