Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models

TL;DR

This paper reviews recent neuro-symbolic approaches that aim to enhance the reasoning abilities of large language models, highlighting methods, challenges, and future directions in this promising research area.

Contribution

It provides a comprehensive survey of neuro-symbolic techniques for improving LLM reasoning, formalizes reasoning tasks, and discusses three key methodological perspectives.

Findings

Neuro-symbolic methods show promise in enhancing LLM reasoning.

The survey categorizes approaches into three perspectives: Symbolic->LLM, LLM->Symbolic, and LLM+Symbolic.

Identifies key challenges and future research directions in neuro-symbolic reasoning.

Abstract

Large Language Models (LLMs) have shown promising results across various tasks, yet their reasoning capabilities remain a fundamental challenge. Developing AI systems with strong reasoning capabilities is regarded as a crucial milestone in the pursuit of Artificial General Intelligence (AGI) and has garnered considerable attention from both academia and industry. Various techniques have been explored to enhance the reasoning capabilities of LLMs, with neuro-symbolic approaches being a particularly promising way. This paper comprehensively reviews recent developments in neuro-symbolic approaches for enhancing LLM reasoning. We first present a formalization of reasoning tasks and give a brief introduction to the neurosymbolic learning paradigm. Then, we discuss neuro-symbolic methods for improving the reasoning capabilities of LLMs from three perspectives: Symbolic->LLM, LLM->Symbolic, and LLM+Symbolic. Finally, we discuss several key challenges and promising future directions. We have also released a GitHub repository including papers and resources related to this survey: https://github.com/LAMDASZ-ML/Awesome-LLM-Reasoning-with-NeSy.