Generating Creative Chess Puzzles

TL;DR

This paper introduces a reinforcement learning framework with novel rewards to significantly improve the generation of creative, diverse, and counter-intuitive chess puzzles, surpassing existing methods and human expert evaluations.

Contribution

The paper presents a new RL-based approach with specialized rewards that enhances the creativity and diversity of AI-generated chess puzzles beyond previous datasets and models.

Findings

Counter-intuitive puzzle generation increased 10x to 2.5%

Puzzles rated more creative and enjoyable by human experts

Achieved diversity and novelty benchmarks surpassing existing datasets

Abstract

While Generative AI rapidly advances in various domains, generating truly creative, aesthetic, and counter-intuitive outputs remains a challenge. This paper presents an approach to tackle these difficulties in the domain of chess puzzles. We start by benchmarking Generative AI architectures, and then introduce an RL framework with novel rewards based on chess engine search statistics to overcome some of those shortcomings. The rewards are designed to enhance a puzzle's uniqueness, counter-intuitiveness, diversity, and realism. Our RL approach dramatically increases counter-intuitive puzzle generation by 10x, from 0.22\% (supervised) to 2.5\%, surpassing existing dataset rates (2.1\%) and the best Lichess-trained model (0.4\%). Our puzzles meet novelty and diversity benchmarks, retain aesthetic themes, and are rated by human experts as more creative, enjoyable, and counter-intuitive than composed book puzzles, even approaching classic compositions. Our final outcome is a curated booklet of these AI-generated puzzles, which is acknowledged for creativity by three world-renowned experts.