Attribute CNNs for Word Spotting in Handwritten Documents

TL;DR

This paper introduces Attribute CNNs for handwritten word spotting, achieving state-of-the-art results by learning attribute representations with CNNs and end-to-end training.

Contribution

It presents novel CNN architectures and loss functions for attribute-based word spotting, advancing beyond previous SVM-based methods.

Findings

Achieved state-of-the-art segmentation-based word spotting results

Demonstrated effectiveness of end-to-end CNN training

Compared different word string embeddings and optimization strategies

Abstract

Word spotting has become a field of strong research interest in document image analysis over the last years. Recently, AttributeSVMs were proposed which predict a binary attribute representation. At their time, this influential method defined the state-of-the-art in segmentation-based word spotting. In this work, we present an approach for learning attribute representations with Convolutional Neural Networks (CNNs). By taking a probabilistic perspective on training CNNs, we derive two different loss functions for binary and real-valued word string embeddings. In addition, we propose two different CNN architectures, specifically designed for word spotting. These architectures are able to be trained in an end-to-end fashion. In a number of experiments, we investigate the influence of different word string embeddings and optimization strategies. We show our Attribute CNNs to achieve state-of-the-art results for segmentation-based word spotting on a large variety of data sets.