A Generic Hybrid Framework for 2D Visual Reconstruction

TL;DR

This paper introduces a flexible hybrid framework combining deep learning and genetic algorithms to improve 2D real-world puzzle reconstruction, achieving state-of-the-art results across various challenging domains.

Contribution

It presents a novel hybrid approach that integrates deep learning-based compatibility measures with genetic algorithms for enhanced puzzle reconstruction.

Findings

Achieves state-of-the-art results in Portuguese tile panel reconstruction.

Demonstrates robustness across multiple real-world puzzle domains.

Effectively handles degraded puzzles with eroded boundaries.

Abstract

This paper presents a versatile hybrid framework for addressing 2D real-world reconstruction tasks formulated as jigsaw puzzle problems (JPPs) with square, non-overlapping pieces. Our approach integrates a deep learning (DL)-based compatibility measure (CM) model that evaluates pairs of puzzle pieces holistically, rather than focusing solely on their adjacent edges as traditionally done. This DL-based CM is paired with an optimized genetic algorithm (GA)-based solver, which iteratively searches for a global optimal arrangement using the pairwise CM scores of the puzzle pieces. Extensive experimental results highlight the framework's adaptability and robustness across multiple real-world domains. Notably, our unique hybrid methodology achieves state-of-the-art (SOTA) results in reconstructing Portuguese tile panels and large degraded puzzles with eroded boundaries.