QirK: Question Answering via Intermediate Representation on Knowledge Graphs

TL;DR

QirK is a system that answers complex natural language questions on Knowledge Graphs by combining database tech, LLMs, and semantic search through an intermediate representation, surpassing some LLM capabilities.

Contribution

It introduces an intermediate representation approach that effectively integrates LLMs, database technology, and semantic search for improved KG question answering.

Findings

Handles structurally complex questions beyond LLMs

Combines LLMs with database and semantic search techniques

Demonstrates practical effectiveness through a system and video

Abstract

We demonstrate QirK, a system for answering natural language questions on Knowledge Graphs (KG). QirK can answer structurally complex questions that are still beyond the reach of emerging Large Language Models (LLMs). It does so using a unique combination of database technology, LLMs, and semantic search over vector embeddings. The glue for these components is an intermediate representation (IR). The input question is mapped to IR using LLMs, which is then repaired into a valid relational database query with the aid of a semantic search on vector embeddings. This allows a practical synthesis of LLM capabilities and KG reliability. A short video demonstrating QirK is available at https://youtu.be/6c81BLmOZ0U.