Language Models are Crossword Solvers

TL;DR

This paper demonstrates that large language models can effectively solve cryptic and full crossword puzzles, outperforming previous methods and achieving high accuracy on real-world puzzles.

Contribution

It introduces a novel approach using LLMs for solving entire crossword grids and shows their strong generalization and reasoning capabilities.

Findings

LLMs outperform previous state-of-the-art in crossword clue solving

Achieved 93% accuracy on New York Times crossword puzzles

LLMs can generate sound rationale supporting answers

Abstract

Crosswords are a form of word puzzle that require a solver to demonstrate a high degree of proficiency in natural language understanding, wordplay, reasoning, and world knowledge, along with adherence to character and length constraints. In this paper we tackle the challenge of solving crosswords with large language models (LLMs). We demonstrate that the current generation of language models shows significant competence at deciphering cryptic crossword clues and outperforms previously reported state-of-the-art (SoTA) results by a factor of 2-3 in relevant benchmarks. We also develop a search algorithm that builds off this performance to tackle the problem of solving full crossword grids with out-of-the-box LLMs for the very first time, achieving an accuracy of 93% on New York Times crossword puzzles. Additionally, we demonstrate that LLMs generalize well and are capable of supporting answers with sound rationale.