Using Graph Neural Networks to Reconstruct Ancient Documents

TL;DR

This paper introduces a graph neural network approach for reconstructing ancient documents by classifying spatial relationships between image patches, enabling the generation of reconstruction graphs from fragments.

Contribution

It presents a novel GNN-based method for ancient document reconstruction that leverages pairwise patch relationships to produce accurate spatial arrangements.

Findings

Accurately classifies spatial relationships between patches

Generates partial or full reconstructions from fragments

Outperforms baseline methods in reconstruction tasks

Abstract

In recent years, machine learning and deep learning approaches such as artificial neural networks have gained in popularity for the resolution of automatic puzzle resolution problems. Indeed, these methods are able to extract high-level representations from images, and then can be trained to separate matching image pieces from non-matching ones. These applications have many similarities to the problem of ancient document reconstruction from partially recovered fragments. In this work we present a solution based on a Graph Neural Network, using pairwise patch information to assign labels to edges representing the spatial relationships between pairs. This network classifies the relationship between a source and a target patch as being one of Up, Down, Left, Right or None. By doing so for all edges, our model outputs a new graph representing a reconstruction proposal. Finally, we show that our model is not only able to provide correct classifications at the edge-level, but also to generate partial or full reconstruction graphs from a set of patches.