Back-of-the-Book Index Automation for Arabic Documents

TL;DR

This paper presents an automated method for verifying and identifying index term occurrences in Arabic books, using NLP techniques and similarity metrics, achieving high accuracy and facilitating index creation and review.

Contribution

It introduces a novel automated approach for back-of-the-book index verification in Arabic, combining noun phrase extraction, vector similarity, and heuristic scoring.

Findings

Achieved an F1-score of 0.966 in index term occurrence identification.

Demonstrated effective use of lexical and semantic similarity metrics.

Facilitated automation in Arabic book indexing processes.

Abstract

Back-of-the-book indexes are crucial for book readability. Their manual creation is laborious and error prone. In this paper, we consider automating back-of-the-book index extraction for Arabic books to help simplify both the creation and review tasks. Given a back-of-the-book index, we aim to check and identify the accurate occurrences of index terms relative to the associated pages. To achieve this, we first define a pool of candidates for each term by extracting all possible noun phrases from paragraphs appearing on the relevant index pages. These noun phrases, identified through part-of-speech analysis, are stored in a vector database for efficient retrieval. We use several metrics, including exact matches, lexical similarity, and semantic similarity, to determine the most appropriate occurrence. The candidate with the highest score based on these metrics is chosen as the occurrence of the term. We fine-tuned a heuristic method, that considers the above metrics and that achieves an F1-score of .966 (precision=.966, recall=.966). These excellent results open the door for future work related to automation of back-of-the-book index generation and checking.