SQuARE: Semantics-based Question Answering and Reasoning Engine

TL;DR

SQuARE is a semantics-based question answering system that uses denotational semantics and answer set programming to understand text and reason about answers without training, achieving perfect accuracy on bAbI datasets.

Contribution

The paper introduces a novel semantics-based framework for QA using denotational semantics and ASP, enabling reasoning without training and providing explanations for answers.

Findings

Achieved 100% accuracy on all tested bAbI QA datasets.

Operates purely on commonsense reasoning without machine learning training.

Can generate explanations for its answers.

Abstract

Understanding the meaning of a text is a fundamental challenge of natural language understanding (NLU) and from its early days, it has received significant attention through question answering (QA) tasks. We introduce a general semantics-based framework for natural language QA and also describe the SQuARE system, an application of this framework. The framework is based on the denotational semantics approach widely used in programming language research. In our framework, valuation function maps syntax tree of the text to its commonsense meaning represented using basic knowledge primitives (the semantic algebra) coded using answer set programming (ASP). We illustrate an application of this framework by using VerbNet primitives as our semantic algebra and a novel algorithm based on partial tree matching that generates an answer set program that represents the knowledge in the text. A question posed against that text is converted into an ASP query using the same framework and executed using the s(CASP) goal-directed ASP system. Our approach is based purely on (commonsense) reasoning. SQuARE achieves 100% accuracy on all the five datasets of bAbI QA tasks that we have tested. The significance of our work is that, unlike other machine learning based approaches, ours is based on "understanding" the text and does not require any training. SQuARE can also generate an explanation for an answer while maintaining high accuracy.