AutoFormBench: Benchmark Dataset for Automating Form Understanding

TL;DR

AutoFormBench is a new benchmark dataset for training and evaluating models on form element detection in diverse real-world documents, addressing layout variability challenges.

Contribution

The paper introduces AutoFormBench, a comprehensive dataset and systematic comparison of classical and deep learning methods for form element detection.

Findings

YOLOv11 outperforms other architectures in accuracy and F1 score.

AutoFormBench covers 407 real-world forms across multiple domains.

Classical OpenCV methods are compared with four YOLO architectures.

Abstract

Automated processing of structured documents such as government forms, healthcare records, and enterprise invoices remains a persistent challenge due to the high degree of layout variability encountered in real-world settings. This paper introduces AutoFormBench, a benchmark dataset of 407 annotated real-world forms spanning government, healthcare, and enterprise domains, designed to train and evaluate form element detection models. We present a systematic comparison of classical OpenCV approaches and four YOLO architectures (YOLOv8, YOLOv11, YOLOv26-s, and YOLOv26-l) for localizing and classifying fillable form elements. specifically checkboxes, input lines, and text boxes across diverse PDF document types. YOLOv11 demonstrates consistently superior performance in both F1 score and Jaccard accuracy across all element classes and tolerance levels.