JigsawNet: Shredded Image Reassembly using Convolutional Neural Network and Loop-based Composition

TL;DR

This paper introduces a deep learning-based method for reassembling shredded images, improving accuracy and reliability over traditional handcrafted feature methods, especially on complex puzzles with many fragments.

Contribution

It presents a novel CNN-based compatibility detector and loop closure algorithms for global composition, advancing shredded image reassembly techniques.

Findings

Outperforms existing methods on various puzzles

Handles complex puzzles with many fragments effectively

Significantly improves reassembly accuracy

Abstract

This paper proposes a novel algorithm to reassemble an arbitrarily shredded image to its original status. Existing reassembly pipelines commonly consist of a local matching stage and a global compositions stage. In the local stage, a key challenge in fragment reassembly is to reliably compute and identify correct pairwise matching, for which most existing algorithms use handcrafted features, and hence, cannot reliably handle complicated puzzles. We build a deep convolutional neural network to detect the compatibility of a pairwise stitching, and use it to prune computed pairwise matches. To improve the network efficiency and accuracy, we transfer the calculation of CNN to the stitching region and apply a boost training strategy. In the global composition stage, we modify the commonly adopted greedy edge selection strategies to two new loop closure based searching algorithms. Extensive experiments show that our algorithm significantly outperforms existing methods on solving various puzzles, especially those challenging ones with many fragment pieces.