Making New Connections: LLMs as Puzzle Generators for The New York Times' Connections Word Game

TL;DR

This paper explores how large language models can generate challenging and creative word puzzles for the NYT Connections game, demonstrating their potential as puzzle creators through human evaluations.

Contribution

It introduces a novel method using Tree of Thoughts prompting to generate puzzles and evaluates their quality via user studies, highlighting LLMs' creative capabilities.

Findings

LLMs can generate diverse, challenging puzzles.

Human players find AI-generated puzzles enjoyable.

The proposed method effectively models solver reasoning.

Abstract

The Connections puzzle is a word association game published daily by The New York Times (NYT). In this game, players are asked to find groups of four words that are connected by a common theme. While solving a given Connections puzzle requires both semantic knowledge and abstract reasoning, generating novel puzzles additionally requires a form of metacognition: generators must be able to accurately model the downstream reasoning of potential solvers. In this paper, we investigate the ability of the GPT family of Large Language Models (LLMs) to generate challenging and creative word games for human players. We start with an analysis of the word game Connections and the unique challenges it poses as a Procedural Content Generation (PCG) domain. We then propose a method for generating Connections puzzles using LLMs by adapting a Tree of Thoughts (ToT) prompting approach. We evaluate this method by conducting a user study, asking human players to compare AI-generated puzzles against published Connections puzzles. Our findings show that LLMs are capable puzzle creators, and can generate diverse sets of enjoyable, challenging, and creative Connections puzzles as judged by human users.