SepFormer: Coarse-to-fine Separator Regression Network for Table Structure Recognition

TL;DR

SepFormer is a novel transformer-based model that efficiently recognizes table structures from images by predicting separators in a coarse-to-fine manner, achieving high speed and competitive accuracy.

Contribution

It introduces a unified, coarse-to-fine separator regression approach with a DETR-style architecture for improved table structure recognition.

Findings

Runs at 25.6 FPS on average

Achieves comparable performance with state-of-the-art methods

Effective on multiple benchmark datasets

Abstract

The automated reconstruction of the logical arrangement of tables from image data, termed Table Structure Recognition (TSR), is fundamental for semantic data extraction. Recently, researchers have explored a wide range of techniques to tackle this problem, demonstrating significant progress. Each table is a set of vertical and horizontal separators. Following this realization, we present SepFormer, which integrates the split-and-merge paradigm into a single step through separator regression with a DETR-style architecture, improving speed and robustness. SepFormer is a coarse-to-fine approach that predicts table separators from single-line to line-strip separators with a stack of two transformer decoders. In the coarse-grained stage, the model learns to gradually refine single-line segments through decoder layers with additional angle loss. At the end of the fine-grained stage, the model predicts line-strip separators by refining sampled points from each single-line segment. Our SepFormer can run on average at 25.6 FPS while achieving comparable performance with state-of-the-art methods on several benchmark datasets, including SciTSR, PubTabNet, WTW, and iFLYTAB.