Riddle Quest : The Enigma of Words

TL;DR

This paper presents a pipeline for generating and evaluating analogy-based riddles, using language models to analyze reasoning coverage and ambiguity in understanding figurative clues.

Contribution

It introduces a novel system for creating and validating riddles, enabling analysis of language models' ability to interpret figurative language and multiple valid answers.

Findings

Language models often identify the main answer but miss alternative valid interpretations.

The riddle generation pipeline can produce diverse analogy-based puzzles.

Riddles serve as effective tools for testing reasoning and ambiguity in NLP models.

Abstract

Riddles are concise linguistic puzzles that describe an object or idea through indirect, figurative, or playful clues. They are a longstanding form of creative expression, requiring the solver to interpret hints, recognize patterns, and draw inferences to identify the answers. In this work, we introduce a simple pipeline for creating and evaluating analogy-based riddles. The system includes a triples creator that builds structured facts about a concept, a semantic mapper that selects attributes useful for analogy, a stylized generator that turns them into riddle clues, and a validator that collects all possible answers the riddle could point to. We use this validator to study whether large language models can recover the full answer set for different riddle types. Our case study shows that while models often guess the main intended answer, they frequently miss other valid interpretations. This highlights the value of riddles as a lightweight tool for examining reasoning coverage and ambiguity handling in language models.