Jigsaw Puzzle Solving Using Local Feature Co-Occurrences in Deep Neural Networks

TL;DR

This paper presents a deep learning-based classification approach for solving jigsaw puzzles by analyzing local feature co-occurrences, achieving a 25% improvement over previous methods and introducing a new dataset from the Metropolitan Museum of Art.

Contribution

The paper introduces a novel deep learning method for classifying relative fragment positions in jigsaw puzzles and a new dataset for archaeological fragment reconstruction.

Findings

Outperforms state-of-the-art by 25% in puzzle classification accuracy

Introduces a new dataset from the Metropolitan Museum of Art

Proposes a greedy reconstruction algorithm based on predicted relations

Abstract

Archaeologists are in dire need of automated object reconstruction methods. Fragments reassembly is close to puzzle problems, which may be solved by computer vision algorithms. As they are often beaten on most image related tasks by deep learning algorithms, we study a classification method that can solve jigsaw puzzles. In this paper, we focus on classifying the relative position: given a couple of fragments, we compute their local relation (e.g. on top). We propose several enhancements over the state of the art in this domain, which is outperformed by our method by 25\%. We propose an original dataset composed of pictures from the Metropolitan Museum of Art. We propose a greedy reconstruction method based on the predicted relative positions.