ODA: Observation-Driven Agent for integrating LLMs and Knowledge Graphs

TL;DR

This paper introduces ODA, a novel AI agent framework that enhances LLM and KG integration by incorporating recursive observation and reasoning, leading to significant accuracy improvements in NLP tasks.

Contribution

The paper presents a new observation-driven agent framework that effectively combines KG reasoning with LLMs, addressing knowledge explosion and improving task performance.

Findings

Achieves state-of-the-art results on multiple datasets.

Improves accuracy by up to 12.87%.

Introduces recursive observation mechanism for better knowledge integration.

Abstract

The integration of Large Language Models (LLMs) and knowledge graphs (KGs) has achieved remarkable success in various natural language processing tasks. However, existing methodologies that integrate LLMs and KGs often navigate the task-solving process solely based on the LLM's analysis of the question, overlooking the rich cognitive potential inherent in the vast knowledge encapsulated in KGs. To address this, we introduce Observation-Driven Agent (ODA), a novel AI agent framework tailored for tasks involving KGs. ODA incorporates KG reasoning abilities via global observation, which enhances reasoning capabilities through a cyclical paradigm of observation, action, and reflection. Confronting the exponential explosion of knowledge during observation, we innovatively design a recursive observation mechanism. Subsequently, we integrate the observed knowledge into the action and reflection modules. Through extensive experiments, ODA demonstrates state-of-the-art performance on several datasets, notably achieving accuracy improvements of 12.87% and 8.9%.