Semantic Segmentation for Compound figures

TL;DR

This paper introduces a semantic segmentation method for decomposing compound figures in scientific literature into individual subfigures, facilitating better information retrieval and analysis.

Contribution

It presents an anchor-based detection algorithm that combines global layout and local features to accurately identify master images within compound figures.

Findings

Effective separation of compound figures demonstrated on a labeled dataset.

Improved accuracy over baseline methods in master image detection.

Facilitates linking subfigures with caption information.

Abstract

Scientific literature contains large volumes of unstructured data,with over 30\% of figures constructed as a combination of multiple images, these compound figures cannot be analyzed directly with existing information retrieval tools. In this paper, we propose a semantic segmentation approach for compound figure separation, decomposing the compound figures into "master images". Each master image is one part of a compound figure governed by a subfigure label (typically "(a), (b), (c), etc"). In this way, the separated subfigures can be easily associated with the description information in the caption. In particular, we propose an anchor-based master image detection algorithm, which leverages the correlation between master images and subfigure labels and locates the master images in a two-step manner. First, a subfigure label detector is built to extract the global layout information of the compound figure. Second, the layout information is combined with local features to locate the master images. We validate the effectiveness of proposed method on our labeled testing dataset both quantitatively and qualitatively.