DANA: Domain-Aware Neurosymbolic Agents for Consistency and Accuracy

TL;DR

DANA is a neurosymbolic architecture that integrates domain knowledge with LLMs to improve consistency and accuracy in complex problem-solving tasks, outperforming existing systems.

Contribution

The paper introduces DANA, a novel neurosymbolic framework that incorporates domain expertise to enhance reliability and precision in AI problem-solving.

Findings

Achieves over 90% accuracy on FinanceBench benchmark

Outperforms current LLM-based systems in consistency and accuracy

Effective in physical industries like semiconductor manufacturing

Abstract

Large Language Models (LLMs) have shown remarkable capabilities, but their inherent probabilistic nature often leads to inconsistency and inaccuracy in complex problem-solving tasks. This paper introduces DANA (Domain-Aware Neurosymbolic Agent), an architecture that addresses these issues by integrating domain-specific knowledge with neurosymbolic approaches. We begin by analyzing current AI architectures, including AutoGPT, LangChain ReAct and OpenAI's ChatGPT, through a neurosymbolic lens, highlighting how their reliance on probabilistic inference contributes to inconsistent outputs. In response, DANA captures and applies domain expertise in both natural-language and symbolic forms, enabling more deterministic and reliable problem-solving behaviors. We implement a variant of DANA using Hierarchical Task Plans (HTPs) in the open-source OpenSSA framework. This implementation achieves over 90\% accuracy on the FinanceBench financial-analysis benchmark, significantly outperforming current LLM-based systems in both consistency and accuracy. Application of DANA in physical industries such as semiconductor shows that its flexible architecture for incorporating knowledge is effective in mitigating the probabilistic limitations of LLMs and has potential in tackling complex, real-world problems that require reliability and precision.