TFLOP: Table Structure Recognition Framework with Layout Pointer Mechanism

TL;DR

TFLOP is a novel table structure recognition framework that directly predicts text regions using a layout pointer mechanism, improving accuracy and efficiency over previous methods, especially in complex and industrial scenarios.

Contribution

The paper introduces TFLOP, a new TSR framework that reformulates text region prediction as a direct pointing task, eliminating the need for region matching and alignment.

Findings

Achieves state-of-the-art results on PubTabNet, FinTabNet, and SynthTabNet.

Performs well on industrial documents with watermarks and non-English content.

Enhances table recognition accuracy with span-aware contrastive supervision.

Abstract

Table Structure Recognition (TSR) is a task aimed at converting table images into a machine-readable format (e.g. HTML), to facilitate other applications such as information retrieval. Recent works tackle this problem by identifying the HTML tags and text regions, where the latter is used for text extraction from the table document. These works however, suffer from misalignment issues when mapping text into the identified text regions. In this paper, we introduce a new TSR framework, called TFLOP (TSR Framework with LayOut Pointer mechanism), which reformulates the conventional text region prediction and matching into a direct text region pointing problem. Specifically, TFLOP utilizes text region information to identify both the table's structure tags and its aligned text regions, simultaneously. Without the need for region prediction and alignment, TFLOP circumvents the additional text region matching stage, which requires finely-calibrated post-processing. TFLOP also employs span-aware contrastive supervision to enhance the pointing mechanism in tables with complex structure. As a result, TFLOP achieves the state-of-the-art performance across multiple benchmarks such as PubTabNet, FinTabNet, and SynthTabNet. In our extensive experiments, TFLOP not only exhibits competitive performance but also shows promising results on industrial document TSR scenarios such as documents with watermarks or in non-English domain.