PuzLM: Solving Jigsaw Puzzles with Sequence-to-Sequence Language Models

TL;DR

PuzLM reimagines jigsaw puzzle solving as a sequence-to-sequence language modeling task, using symbolic representations of pieces to achieve accurate reconstruction, even with challenging puzzle conditions.

Contribution

This work introduces a novel Seq2Seq approach for puzzle solving by transforming pieces into discrete tokens, enabling symbolic reasoning for improved accuracy.

Findings

Achieves state-of-the-art performance on complex puzzles

Handles eroded boundaries and missing pieces effectively

Demonstrates the power of language models in visual puzzle solving

Abstract

Square jigsaw puzzles are typically solved by visually matching piece images to recover the original layout. This work introduces PuzLM, an alternative perspective that recasts jigsaw reassembly as a discrete sequence-to-sequence (Seq2Seq) problem, inspired by natural language representations. We design an efficient puzzle quantization procedure that transforms each piece into a short sequence of discrete tokens, enabling the direct application of standard Seq2Seq language models as powerful jigsaw solvers. Our approach demonstrates that accurate puzzle reconstruction can be achieved through purely symbolic reasoning over discrete representations, improving state-of-the-art performance even on puzzles with eroded boundaries or missing pieces.