Pandora: Leveraging Code-driven Knowledge Transfer for Unified Structured Knowledge Reasoning

TL;DR

Pandora introduces a code-driven unified knowledge representation using Python's Pandas API, enabling large language models to perform cross-task structured knowledge reasoning more effectively through knowledge transfer and adaptive feedback.

Contribution

It presents a novel framework that leverages code-based representations and knowledge transfer to improve unified structured knowledge reasoning across multiple tasks.

Findings

Outperforms existing unified reasoning frameworks on six benchmarks.

Effectively handles cross-task knowledge transfer and reasoning correction.

Achieves competitive results with task-specific methods.

Abstract

Unified Structured Knowledge Reasoning (USKR) aims to answer natural language questions by using structured sources such as tables, databases, and knowledge graphs in a unified way. Existing USKR methods rely on task-specific strategies or bespoke representations, which hinder their ability to dismantle barriers between different SKR tasks, thereby constraining their overall performance in cross-task scenarios. In this paper, we introduce \textsc{Pandora}, a novel USKR framework that addresses the limitations of existing methods by leveraging two key innovations. First, we propose a code-based unified knowledge representation using \textsc{Python}'s \textsc{Pandas} API, which aligns seamlessly with the pre-training of LLMs. This representation facilitates a cohesive approach to handling different structured knowledge sources. Building on this foundation, we employ knowledge transfer to bolster the unified reasoning process of LLMs by automatically building cross-task memory. By adaptively correcting reasoning using feedback from code execution, \textsc{Pandora} showcases impressive unified reasoning capabilities. Extensive experiments on six widely used benchmarks across three SKR tasks demonstrate that \textsc{Pandora} outperforms existing unified reasoning frameworks and competes effectively with task-specific methods.