Improving Chess Commentaries by Combining Language Models with Symbolic Reasoning Engines

TL;DR

This paper presents a method that combines symbolic reasoning engines with controllable language models to generate more accurate and human-preferred chess commentaries, bridging complex reasoning and natural language generation.

Contribution

It introduces a novel approach integrating symbolic reasoning with language models for chess commentary, enhancing reasoning and natural language quality.

Findings

Generated commentaries are preferred by human judges.

The approach outperforms previous baselines.

Demonstrates effective integration of reasoning and language models.

Abstract

Despite many recent advancements in language modeling, state-of-the-art language models lack grounding in the real world and struggle with tasks involving complex reasoning. Meanwhile, advances in the symbolic reasoning capabilities of AI have led to systems that outperform humans in games like chess and Go (Silver et al., 2018). Chess commentary provides an interesting domain for bridging these two fields of research, as it requires reasoning over a complex board state and providing analyses in natural language. In this work we demonstrate how to combine symbolic reasoning engines with controllable language models to generate chess commentaries. We conduct experiments to demonstrate that our approach generates commentaries that are preferred by human judges over previous baselines.