Handling tree-structured text: parsing directory pages

TL;DR

This paper addresses the challenge of parsing irregular, hierarchical directory pages in financial documents by developing classifiers and a bottom-up traversal method to construct reading trees for improved document understanding.

Contribution

It introduces a novel approach for identifying directory pages and constructing reading trees using learned classifiers and hierarchical traversal, enhancing document processing capabilities.

Findings

Successfully identifies directory pages in financial documents.

Constructs reading trees to capture hierarchical structure.

Supports automatic extraction of organizational information.

Abstract

The determination of the reading sequence of text is fundamental to document understanding. This problem is easily solved in pages where the text is organized into a sequence of lines and vertical alignment runs the height of the page (producing multiple columns which can be read from left to right). We present a situation -- the directory page parsing problem -- where information is presented on the page in an irregular, visually-organized, two-dimensional format. Directory pages are fairly common in financial prospectuses and carry information about organizations, their addresses and relationships that is key to business tasks in client onboarding. Interestingly, directory pages sometimes have hierarchical structure, motivating the need to generalize the reading sequence to a reading tree. We present solutions to the problem of identifying directory pages and constructing the reading tree, using (learnt) classifiers for text segments and a bottom-up (right to left, bottom-to-top) traversal of segments. The solution is a key part of a production service supporting automatic extraction of organization, address and relationship information from client onboarding documents.