Logical segmentation for article extraction in digitized old newspapers

TL;DR

This paper presents an automated system for extracting and structuring articles from digitized old newspapers, enabling detailed indexing, retrieval, and collaborative correction to improve access to historical newspaper archives.

Contribution

The authors develop a machine learning-based method to identify logical structures in newspaper pages and integrate this with a web interface for article-level access and correction.

Findings

Successfully applied to 250 years of the Journal de Rouen archives

Achieved pixel-level labeling and structure detection in variable quality images

Enhanced article retrieval and indexing through logical segmentation

Abstract

Newspapers are documents made of news item and informative articles. They are not meant to be red iteratively: the reader can pick his items in any order he fancies. Ignoring this structural property, most digitized newspaper archives only offer access by issue or at best by page to their content. We have built a digitization workflow that automatically extracts newspaper articles from images, which allows indexing and retrieval of information at the article level. Our back-end system extracts the logical structure of the page to produce the informative units: the articles. Each image is labelled at the pixel level, through a machine learning based method, then the page logical structure is constructed up from there by the detection of structuring entities such as horizontal and vertical separators, titles and text lines. This logical structure is stored in a METS wrapper associated to the ALTO file produced by the system including the OCRed text. Our front-end system provides a web high definition visualisation of images, textual indexing and retrieval facilities, searching and reading at the article level. Articles transcriptions can be collaboratively corrected, which as a consequence allows for better indexing. We are currently testing our system on the archives of the Journal de Rouen, one of France eldest local newspaper. These 250 years of publication amount to 300 000 pages of very variable image quality and layout complexity. Test year 1808 can be consulted at plair.univ-rouen.fr.