A Reasoning-Based Approach to Cryptic Crossword Clue Solving

TL;DR

This paper introduces a reasoning-based system using large language models to solve cryptic crossword clues, achieving state-of-the-art results and providing interpretable explanations of the reasoning process.

Contribution

It presents a novel LLM-based reasoning approach that hypothesizes answers, explains wordplay, and verifies solutions, advancing cryptic crossword solving methods.

Findings

Achieves new state-of-the-art performance on Cryptonite dataset

Provides interpretable reasoning steps in Python code

Demonstrates effective hypothesis and verification cycle

Abstract

Cryptic crossword clues are challenging language tasks for which new test sets are released daily by major newspapers on a global basis. Each cryptic clue contains both the definition of the answer to be placed in the crossword grid (in common with regular crosswords), and 'wordplay' that proves that the answer is correct (i.e. a human solver can be confident that an answer is correct without needing crossing words as confirmation). This work describes an LLM-based reasoning system built from open-licensed components that solves cryptic clues by (i) hypothesising answers; (ii) proposing wordplay explanations; and (iii) using a verifier system that operates on codified reasoning steps. Overall, this system establishes a new state-of-the-art performance on the challenging Cryptonite dataset of clues from The Times and The Telegraph newspapers in the UK. Because each proved solution is expressed in Python, interpretable wordplay reasoning for proven answers is available for inspection.