Logic-of-Thought: Empowering Large Language Models with Logic Programs for Solving Puzzles in Natural Language

TL;DR

Logic-of-Thought (Logot) combines large language models with logic programming to solve complex puzzles in natural language with high accuracy, bridging natural language understanding and precise logical reasoning.

Contribution

Introduces Logot, a hybrid framework that translates puzzle rules into answer set programs using LLMs, enabling accurate and efficient puzzle solving.

Findings

Achieves near-perfect accuracy on grid and dynamic puzzles

Effectively combines LLMs with logic programming for reasoning tasks

Demonstrates the effectiveness of hybrid approach in complex puzzle solving

Abstract

Solving puzzles in natural language poses a long-standing challenge in AI. While large language models (LLMs) have recently shown impressive capabilities in a variety of tasks, they continue to struggle with complex puzzles that demand precise reasoning and exhaustive search. In this paper, we propose Logic-of-Thought (Logot), a novel framework that bridges LLMs with logic programming to address this problem. Our method leverages LLMs to translate puzzle rules and states into answer set programs (ASPs), the solution of which are then accurately and efficiently inferred by an ASP interpreter. This hybrid approach combines the natural language understanding of LLMs with the precise reasoning capabilities of logic programs. We evaluate our method on various grid puzzles and dynamic puzzles involving actions, demonstrating near-perfect accuracy across all tasks. Our code and data are available at: https://github.com/naiqili/Logic-of-Thought.