Advancing Symbolic Integration in Large Language Models: Beyond Conventional Neurosymbolic AI

TL;DR

This paper reviews and categorizes methods for integrating symbolic AI into large language models to improve transparency, proposing a new taxonomy and roadmap based on current research, benchmarks, and gaps in the field.

Contribution

It introduces a novel taxonomy and categorization framework for symbolic integration in LLMs, along with a comprehensive roadmap for future research and implementation.

Findings

Identifies key benchmarks and advancements in symbolic integration for LLMs.

Highlights critical gaps and challenges in current approaches.

Provides practical insights and a structured framework for future development.

Abstract

LLMs have demonstrated highly effective learning, human-like response generation,and decision-making capabilities in high-risk sectors. However, these models remain black boxes because they struggle to ensure transparency in responses. The literature has explored numerous approaches to address transparency challenges in LLMs, including Neurosymbolic AI (NeSy AI). NeSy AI approaches were primarily developed for conventional neural networks and are not well-suited to the unique features of LLMs. Consequently, there is a limited systematic understanding of how symbolic AI can be effectively integrated into LLMs. This paper aims to address this gap by first reviewing established NeSy AI methods and then proposing a novel taxonomy of symbolic integration in LLMs, along with a roadmap to merge symbolic techniques with LLMs. The roadmap introduces a new categorisation framework across four dimensions by organising existing literature within these categories. These include symbolic integration across various stages of LLM, coupling mechanisms, architectural paradigms, as well as algorithmic and application-level perspectives. The paper thoroughly identifies current benchmarks, cutting-edge advancements, and critical gaps within the field to propose a roadmap for future research. By highlighting the latest developments and notable gaps in the literature, it offers practical insights for implementing frameworks for symbolic integration into LLMs to enhance transparency.