Message Passing for Complex Question Answering over Knowledge Graphs

TL;DR

This paper introduces an unsupervised message passing method for complex question answering over knowledge graphs, improving accuracy by propagating confidence scores through graph structures and outperforming existing methods on benchmark datasets.

Contribution

The paper presents a novel unsupervised message passing approach for complex KGQA that efficiently propagates confidence scores and outperforms state-of-the-art methods on LC-QuAD.

Findings

Outperforms state-of-the-art on LC-QuAD benchmark

Performance depends on question interpretation quality

Error analysis reveals dataset and knowledge graph issues

Abstract

Question answering over knowledge graphs (KGQA) has evolved from simple single-fact questions to complex questions that require graph traversal and aggregation. We propose a novel approach for complex KGQA that uses unsupervised message passing, which propagates confidence scores obtained by parsing an input question and matching terms in the knowledge graph to a set of possible answers. First, we identify entity, relationship, and class names mentioned in a natural language question, and map these to their counterparts in the graph. Then, the confidence scores of these mappings propagate through the graph structure to locate the answer entities. Finally, these are aggregated depending on the identified question type. This approach can be efficiently implemented as a series of sparse matrix multiplications mimicking joins over small local subgraphs. Our evaluation results show that the proposed approach outperforms the state-of-the-art on the LC-QuAD benchmark. Moreover, we show that the performance of the approach depends only on the quality of the question interpretation results, i.e., given a correct relevance score distribution, our approach always produces a correct answer ranking. Our error analysis reveals correct answers missing from the benchmark dataset and inconsistencies in the DBpedia knowledge graph. Finally, we provide a comprehensive evaluation of the proposed approach accompanied with an ablation study and an error analysis, which showcase the pitfalls for each of the question answering components in more detail.