Multi-Cell Decoder and Mutual Learning for Table Structure and Character Recognition

TL;DR

This paper introduces a multi-cell decoder and mutual learning mechanism to enhance end-to-end table structure and character recognition, enabling better handling of long tables and leveraging neighboring cell information.

Contribution

It proposes a novel multi-cell content decoder and bidirectional mutual learning to improve table recognition accuracy and context utilization in end-to-end models.

Findings

Achieves comparable performance to state-of-the-art models on large datasets.

Effectively recognizes long tables with hundreds of cells.

Improves utilization of neighboring cell information for better recognition.

Abstract

Extracting table contents from documents such as scientific papers and financial reports and converting them into a format that can be processed by large language models is an important task in knowledge information processing. End-to-end approaches, which recognize not only table structure but also cell contents, achieved performance comparable to state-of-the-art models using external character recognition systems, and have potential for further improvements. In addition, these models can now recognize long tables with hundreds of cells by introducing local attention. However, the models recognize table structure in one direction from the header to the footer, and cell content recognition is performed independently for each cell, so there is no opportunity to retrieve useful information from the neighbor cells. In this paper, we propose a multi-cell content decoder and bidirectional mutual learning mechanism to improve the end-to-end approach. The effectiveness is demonstrated on two large datasets, and the experimental results show comparable performance to state-of-the-art models, even for long tables with large numbers of cells.