Benchmarking Content-Based Puzzle Solvers on Corrupted Jigsaw Puzzles

TL;DR

This paper evaluates the robustness of content-based puzzle solvers against realistic corruptions like missing pieces and erosion, highlighting the strengths of deep learning models and proposing future research directions.

Contribution

It introduces a comprehensive benchmarking of puzzle solvers under realistic corruptions and demonstrates the effectiveness of fine-tuning deep learning models, especially the Positional Diffusion model.

Findings

Deep learning models outperform heuristic methods under corruptions.

Performance declines rapidly with increased corrupted pieces.

Fine-tuning with augmented data improves robustness significantly.

Abstract

Content-based puzzle solvers have been extensively studied, demonstrating significant progress in computational techniques. However, their evaluation often lacks realistic challenges crucial for real-world applications, such as the reassembly of fragmented artefacts or shredded documents. In this work, we investigate the robustness of State-Of-The-Art content-based puzzle solvers introducing three types of jigsaw puzzle corruptions: missing pieces, eroded edges, and eroded contents. Evaluating both heuristic and deep learning-based solvers, we analyse their ability to handle these corruptions and identify key limitations. Our results show that solvers developed for standard puzzles have a rapid decline in performance if more pieces are corrupted. However, deep learning models can significantly improve their robustness through fine-tuning with augmented data. Notably, the advanced Positional Diffusion model adapts particularly well, outperforming its competitors in most experiments. Based on our findings, we highlight promising research directions for enhancing the automated reconstruction of real-world artefacts.