TrustUQA: A Trustful Framework for Unified Structured Data Question Answering

TL;DR

TrustUQA is a unified, trustful question answering framework that effectively handles multiple structured data types using a novel Condition Graph representation and demonstrates state-of-the-art performance across various benchmarks.

Contribution

It introduces a unified knowledge representation called Condition Graph and a two-level querying method, enhancing trustfulness and generalization in structured data QA.

Findings

Outperforms existing unified structured data QA methods.

Achieves state-of-the-art results on 2 datasets.

Demonstrates effectiveness on mixed and cross-structured data QA.

Abstract

Natural language question answering (QA) over structured data sources such as tables and knowledge graphs have been widely investigated, especially with Large Language Models (LLMs) in recent years. The main solutions include question to formal query parsing and retrieval-based answer generation. However, current methods of the former often suffer from weak generalization, failing to dealing with multi-types of sources, while the later is limited in trustfulness. In this paper, we propose TrustUQA, a trustful QA framework that can simultaneously support multiple types of structured data in a unified way. To this end, it adopts an LLM-friendly and unified knowledge representation method called Condition Graph(CG), and uses an LLM and demonstration-based two-level method for CG querying. For enhancement, it is also equipped with dynamic demonstration retrieval. We have evaluated TrustUQA with 5 benchmarks covering 3 types of structured data. It outperforms 2 existing unified structured data QA methods. In comparison with the baselines that are specific to one data type, it achieves state-of-the-art on 2 of the datasets. Further more, we have demonstrated the potential of our method for more general QA tasks, QA over mixed structured data and QA across structured data. The code is available at https://github.com/zjukg/TrustUQA.