BoundaryNet: An Attentive Deep Network with Fast Marching Distance Maps for Semi-automatic Layout Annotation

TL;DR

BoundaryNet is a novel semi-automatic deep learning approach that accurately annotates complex document image regions without resizing, leveraging attention mechanisms and fast marching distance maps for high-quality boundary detection.

Contribution

The paper introduces BoundaryNet, a resizing-free, semi-automatic boundary annotation method combining attention-guided networks and fast marching maps, outperforming existing approaches on complex document images.

Findings

Outperforms strong baselines on manuscript datasets

Generalizes across multiple document layouts and scripts

Provides high annotation throughput in practical systems

Abstract

Precise boundary annotations of image regions can be crucial for downstream applications which rely on region-class semantics. Some document collections contain densely laid out, highly irregular and overlapping multi-class region instances with large range in aspect ratio. Fully automatic boundary estimation approaches tend to be data intensive, cannot handle variable-sized images and produce sub-optimal results for aforementioned images. To address these issues, we propose BoundaryNet, a novel resizing-free approach for high-precision semi-automatic layout annotation. The variable-sized user selected region of interest is first processed by an attention-guided skip network. The network optimization is guided via Fast Marching distance maps to obtain a good quality initial boundary estimate and an associated feature representation. These outputs are processed by a Residual Graph Convolution Network optimized using Hausdorff loss to obtain the final region boundary. Results on a challenging image manuscript dataset demonstrate that BoundaryNet outperforms strong baselines and produces high-quality semantic region boundaries. Qualitatively, our approach generalizes across multiple document image datasets containing different script systems and layouts, all without additional fine-tuning. We integrate BoundaryNet into a document annotation system and show that it provides high annotation throughput compared to manual and fully automatic alternatives.