Effective Search of Logical Forms for Weakly Supervised Knowledge-Based Question Answering

TL;DR

This paper introduces a search method for logical forms in weakly supervised KBQA that uses operator prediction to improve search efficiency and quality, leading to better semantic parsing models.

Contribution

It proposes an operator prediction-based search approach that constrains the search space, increasing valid logical forms and reducing spurious ones in weakly supervised KBQA.

Findings

Improved precision from 67% to 72%.

Enhanced recall from 67% to 72%.

More effective training data generation.

Abstract

Many algorithms for Knowledge-Based Question Answering (KBQA) depend on semantic parsing, which translates a question to its logical form. When only weak supervision is provided, it is usually necessary to search valid logical forms for model training. However, a complex question typically involves a huge search space, which creates two main problems: 1) the solutions limited by computation time and memory usually reduce the success rate of the search, and 2) spurious logical forms in the search results degrade the quality of training data. These two problems lead to a poorly-trained semantic parsing model. In this work, we propose an effective search method for weakly supervised KBQA based on operator prediction for questions. With search space constrained by predicted operators, sufficient search paths can be explored, more valid logical forms can be derived, and operators possibly causing spurious logical forms can be avoided. As a result, a larger proportion of questions in a weakly supervised training set are equipped with logical forms, and fewer spurious logical forms are generated. Such high-quality training data directly contributes to a better semantic parsing model. Experimental results on one of the largest KBQA datasets (i.e., CSQA) verify the effectiveness of our approach: improving the precision from 67% to 72% and the recall from 67% to 72% in terms of the overall score.