Open Evaluation Tool for Layout Analysis of Document Images

TL;DR

This paper introduces an open-source, versatile evaluation tool for pixel-level layout analysis of document images, supporting multiple metrics and visualization, to standardize and facilitate research in this area.

Contribution

It presents a new, freely available evaluation tool that supports multi-labeled ground truth and can be used as a standalone application or web service.

Findings

Successfully used in ICDAR2017 competition

Supports multiple evaluation metrics

Enables easy visualization of results

Abstract

This paper presents an open tool for standardizing the evaluation process of the layout analysis task of document images at pixel level. We introduce a new evaluation tool that is both available as a standalone Java application and as a RESTful web service. This evaluation tool is free and open-source in order to be a common tool that anyone can use and contribute to. It aims at providing as many metrics as possible to investigate layout analysis predictions, and also provide an easy way of visualizing the results. This tool evaluates document segmentation at pixel level, and support multi-labeled pixel ground truth. Finally, this tool has been successfully used for the ICDAR2017 competition on Layout Analysis for Challenging Medieval Manuscripts.