Logical Reasoning in Large Language Models: A Survey

TL;DR

This survey reviews recent progress in enabling large language models to perform rigorous logical reasoning, analyzing their capabilities, evaluation benchmarks, and strategies for improvement across various reasoning paradigms.

Contribution

It provides a comprehensive synthesis of recent advancements, theoretical foundations, and evaluation methods in logical reasoning for large language models, highlighting future research directions.

Findings

LLMs show promising reasoning abilities but still face challenges in rigorous logic tasks.

Various strategies like data tuning and neuro-symbolic methods can enhance reasoning performance.

Benchmark evaluations reveal current limitations and areas for improvement.

Abstract

With the emergence of advanced reasoning models like OpenAI o3 and DeepSeek-R1, large language models (LLMs) have demonstrated remarkable reasoning capabilities. However, their ability to perform rigorous logical reasoning remains an open question. This survey synthesizes recent advancements in logical reasoning within LLMs, a critical area of AI research. It outlines the scope of logical reasoning in LLMs, its theoretical foundations, and the benchmarks used to evaluate reasoning proficiency. We analyze existing capabilities across different reasoning paradigms - deductive, inductive, abductive, and analogical - and assess strategies to enhance reasoning performance, including data-centric tuning, reinforcement learning, decoding strategies, and neuro-symbolic approaches. The review concludes with future directions, emphasizing the need for further exploration to strengthen logical reasoning in AI systems.