DARA: Decomposition-Alignment-Reasoning Autonomous Language Agent for Question Answering over Knowledge Graphs

TL;DR

DARA is a neural-symbolic framework that enhances language agents' ability to answer questions over knowledge graphs by decomposing questions and grounding tasks, achieving high accuracy with limited training data.

Contribution

Introducing DARA, a novel framework that combines question decomposition and grounding for improved KGQA performance with small training sets.

Findings

DARA outperforms in-context learning agents on multiple benchmarks.

DARA achieves performance comparable to state-of-the-art methods.

Efficient training with limited high-quality reasoning trajectories.

Abstract

Answering Questions over Knowledge Graphs (KGQA) is key to well-functioning autonomous language agents in various real-life applications. To improve the neural-symbolic reasoning capabilities of language agents powered by Large Language Models (LLMs) in KGQA, we propose the DecompositionAlignment-Reasoning Agent (DARA) framework. DARA effectively parses questions into formal queries through a dual mechanism: high-level iterative task decomposition and low-level task grounding. Importantly, DARA can be efficiently trained with a small number of high-quality reasoning trajectories. Our experimental results demonstrate that DARA fine-tuned on LLMs (e.g. Llama-2-7B, Mistral) outperforms both in-context learning-based agents with GPT-4 and alternative fine-tuned agents, across different benchmarks in zero-shot evaluation, making such models more accessible for real-life applications. We also show that DARA attains performance comparable to state-of-the-art enumerating-and-ranking-based methods for KGQA.