EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs

TL;DR

EffiQA introduces a three-stage collaborative framework that enhances question-answering over knowledge graphs by balancing reasoning accuracy and computational efficiency through iterative planning, exploration, and self-reflection.

Contribution

The paper presents a novel multi-model collaboration framework that improves efficiency and reasoning in knowledge graph question answering by combining LLMs with a plug-in model in an iterative process.

Findings

Achieves a better balance between accuracy and efficiency on KBQA benchmarks.

Utilizes a three-stage process: planning, exploration, and self-reflection.

Demonstrates significant improvements over existing methods.

Abstract

While large language models (LLMs) have shown remarkable capabilities in natural language processing, they struggle with complex, multi-step reasoning tasks involving knowledge graphs (KGs). Existing approaches that integrate LLMs and KGs either underutilize the reasoning abilities of LLMs or suffer from prohibitive computational costs due to tight coupling. To address these limitations, we propose a novel collaborative framework named EffiQA that can strike a balance between performance and efficiency via an iterative paradigm. EffiQA consists of three stages: global planning, efficient KG exploration, and self-reflection. Specifically, EffiQA leverages the commonsense capability of LLMs to explore potential reasoning pathways through global planning. Then, it offloads semantic pruning to a small plug-in model for efficient KG exploration. Finally, the exploration results are fed to LLMs for self-reflection to further improve the global planning and efficient KG exploration. Empirical evidence on multiple KBQA benchmarks shows EffiQA's effectiveness, achieving an optimal balance between reasoning accuracy and computational costs. We hope the proposed new framework will pave the way for efficient, knowledge-intensive querying by redefining the integration of LLMs and KGs, fostering future research on knowledge-based question answering.