Solving Jigsaw Puzzles in the Wild: Human-Guided Reconstruction of Cultural Heritage Fragments
Omidreza Safaei, Sinem Aslan, Sebastiano Vascon, Luca Palmieri, Marina Khoroshiltseva, Marcello Pelillo

TL;DR
This paper presents a human-guided framework for reassembling complex archaeological fragments, combining automatic algorithms with interactive human input to improve accuracy and scalability in real-world artifact reconstruction.
Contribution
It introduces a novel hybrid human-in-the-loop approach with two interaction strategies, enhancing large-scale cultural heritage puzzle solving beyond traditional methods.
Findings
Outperforms automatic and manual baselines in accuracy
Enables scalable reconstruction for large, ambiguous puzzles
Demonstrates effectiveness on RePAIR benchmark groups
Abstract
Reassembling real-world archaeological artifacts from fragmented pieces poses significant challenges due to erosion, missing regions, irregular shapes, and large-scale ambiguity. Traditional jigsaw puzzle solvers, often designed for clean synthetic scenarios, struggle under these conditions, especially when the number of fragments grows into the thousands, as in the RePAIR benchmark. In this paper, we propose a human-in-the-loop (HIL) puzzle solving framework designed to address the complexity and scale of real-world cultural heritage reconstruction. Our approach integrates an automatic relaxation-labeling solver with interactive human guidance, allowing users to iteratively lock verified placements, correct errors, and guide the system toward semantically and geometrically coherent assemblies. We introduce two complementary interaction strategies, Iterative Anchoring and Continuous…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Processing and 3D Reconstruction · 3D Shape Modeling and Analysis · Image and Object Detection Techniques
