Relaxation Labeling Meets GANs: Solving Jigsaw Puzzles with Missing Borders

TL;DR

This paper introduces JiGAN, a novel GAN-based approach that repairs missing borders in jigsaw puzzle pieces and then uses relaxation labeling to accurately assemble them, addressing real-world challenges like artifact reconstruction.

Contribution

The paper presents a two-step method combining GAN-based border repair with relaxation labeling for solving puzzles with missing borders, a novel approach in puzzle reconstruction.

Findings

Effective border repair with GANs improves puzzle assembly accuracy.

Method outperforms existing techniques on benchmark datasets.

Demonstrates feasibility for real-world artifact reconstruction.

Abstract

This paper proposes JiGAN, a GAN-based method for solving Jigsaw puzzles with eroded or missing borders. Missing borders is a common real-world situation, for example, when dealing with the reconstruction of broken artifacts or ruined frescoes. In this particular condition, the puzzle's pieces do not align perfectly due to the borders' gaps; in this situation, the patches' direct match is unfeasible due to the lack of color and line continuations. JiGAN, is a two-steps procedure that tackles this issue: first, we repair the eroded borders with a GAN-based image extension model and measure the alignment affinity between pieces; then, we solve the puzzle with the relaxation labeling algorithm to enforce consistency in pieces positioning, hence, reconstructing the puzzle. We test the method on a large dataset of small puzzles and on three commonly used benchmark datasets to demonstrate the feasibility of the proposed approach.