ICFHR 2020 Competition on Image Retrieval for Historical Handwritten Fragments

TL;DR

This paper presents a large-scale competition on image retrieval for historical handwritten fragments, focusing on style and writer identification, with a new dataset and analysis of method performance.

Contribution

It introduces a new dataset of over 120,000 fragments and evaluates retrieval methods, highlighting challenges and insights in historical document analysis.

Findings

Winning method achieved an mAP below 40%

Deep learning methods dominate submissions

Sample granularity impacts retrieval performance

Abstract

This competition succeeds upon a line of competitions for writer and style analysis of historical document images. In particular, we investigate the performance of large-scale retrieval of historical document fragments in terms of style and writer identification. The analysis of historic fragments is a difficult challenge commonly solved by trained humanists. In comparison to previous competitions, we make the results more meaningful by addressing the issue of sample granularity and moving from writer to page fragment retrieval. The two approaches, style and author identification, provide information on what kind of information each method makes better use of and indirectly contribute to the interpretability of the participating method. Therefore, we created a large dataset consisting of more than 120 000 fragments. Although the most teams submitted methods based on convolutional neural networks, the winning entry achieves an mAP below 40%.