Latent Diffusion for Guided Document Table Generation

TL;DR

This paper presents a novel latent diffusion-based method for generating annotated document table images, improving training data quality for table structure recognition models and achieving performance comparable to state-of-the-art methods.

Contribution

The study introduces a conditioned latent diffusion approach to generate realistic annotated table images, enhancing synthetic data quality for training object detection models.

Findings

Generated data improves YOLOv5 performance on pubtables-1m.

Synthetic images achieve low FID scores, indicating high realism.

Method yields results comparable to state-of-the-art in table recognition.

Abstract

Obtaining annotated table structure data for complex tables is a challenging task due to the inherent diversity and complexity of real-world document layouts. The scarcity of publicly available datasets with comprehensive annotations for intricate table structures hinders the development and evaluation of models designed for such scenarios. This research paper introduces a novel approach for generating annotated images for table structure by leveraging conditioned mask images of rows and columns through the application of latent diffusion models. The proposed method aims to enhance the quality of synthetic data used for training object detection models. Specifically, the study employs a conditioning mechanism to guide the generation of complex document table images, ensuring a realistic representation of table layouts. To evaluate the effectiveness of the generated data, we employ the popular YOLOv5 object detection model for training. The generated table images serve as valuable training samples, enriching the dataset with diverse table structures. The model is subsequently tested on the challenging pubtables-1m testset, a benchmark for table structure recognition in complex document layouts. Experimental results demonstrate that the introduced approach significantly improves the quality of synthetic data for training, leading to YOLOv5 models with enhanced performance. The mean Average Precision (mAP) values obtained on the pubtables-1m testset showcase results closely aligned with state-of-the-art methods. Furthermore, low FID results obtained on the synthetic data further validate the efficacy of the proposed methodology in generating annotated images for table structure.