Historical Document Image Segmentation with LDA-Initialized Deep Neural Networks

TL;DR

This paper introduces a novel LDA-based weight initialization method for deep neural networks, specifically applied to historical document image segmentation, demonstrating faster training and improved accuracy over traditional methods.

Contribution

The paper presents a new LDA-based initialization technique for neural networks, enhancing training stability and performance in historical document segmentation tasks.

Findings

LDA initialization is quick and stable.

LDA-based initialization outperforms random methods.

Improves layout analysis accuracy.

Abstract

In this paper, we present a novel approach to perform deep neural networks layer-wise weight initialization using Linear Discriminant Analysis (LDA). Typically, the weights of a deep neural network are initialized with: random values, greedy layer-wise pre-training (usually as Deep Belief Network or as auto-encoder) or by re-using the layers from another network (transfer learning). Hence, many training epochs are needed before meaningful weights are learned, or a rather similar dataset is required for seeding a fine-tuning of transfer learning. In this paper, we describe how to turn an LDA into either a neural layer or a classification layer. We analyze the initialization technique on historical documents. First, we show that an LDA-based initialization is quick and leads to a very stable initialization. Furthermore, for the task of layout analysis at pixel level, we investigate the effectiveness of LDA-based initialization and show that it outperforms state-of-the-art random weight initialization methods.