Document Layout Analysis on BaDLAD Dataset: A Comprehensive MViTv2 Based Approach

TL;DR

This paper presents a comprehensive approach to document layout analysis using a MViTv2 transformer model with cascaded mask R-CNN on the BaDLAD dataset, achieving effective extraction of various document components.

Contribution

The study introduces a novel application of MViTv2 transformer architecture combined with cascaded mask R-CNN for detailed document layout analysis on BaDLAD, including extensive exploration of augmentation and inference techniques.

Findings

Rotation and flip augmentation can improve detection accuracy.

Input image slicing affects model performance.

Varying transformer backbone resolution impacts results.

Abstract

In the rapidly evolving digital era, the analysis of document layouts plays a pivotal role in automated information extraction and interpretation. In our work, we have trained MViTv2 transformer model architecture with cascaded mask R-CNN on BaDLAD dataset to extract text box, paragraphs, images and tables from a document. After training on 20365 document images for 36 epochs in a 3 phase cycle, we achieved a training loss of 0.2125 and a mask loss of 0.19. Our work extends beyond training, delving into the exploration of potential enhancement avenues. We investigate the impact of rotation and flip augmentation, the effectiveness of slicing input images pre-inference, the implications of varying the resolution of the transformer backbone, and the potential of employing a dual-pass inference to uncover missed text-boxes. Through these explorations, we observe a spectrum of outcomes, where some modifications result in tangible performance improvements, while others offer unique insights for future endeavors.