Distantly Supervised Question Parsing

TL;DR

This paper introduces a reinforcement learning-based framework for distantly supervised question parsing that improves entity and relation linking, thereby enhancing QA system performance without requiring gold annotations.

Contribution

It presents a novel distantly supervised learning approach using reinforcement learning to improve question parsing in QA systems, leveraging formal queries for training.

Findings

Significant improvement in entity and relation linking accuracy.

Enhanced overall QA system performance.

Effective use of formal queries as distant supervision signals.

Abstract

The emergence of structured databases for Question Answering (QA) systems has led to developing methods, in which the problem of learning the correct answer efficiently is based on a linking task between the constituents of the question and the corresponding entries in the database. As a result, parsing the questions in order to determine their main elements, which are required for answer retrieval, becomes crucial. However, most datasets for QA systems lack gold annotations for parsing, i.e., labels are only available in the form of (question, formal-query, answer). In this paper, we propose a distantly supervised learning framework based on reinforcement learning to learn the mentions of entities and relations in questions. We leverage the provided formal queries to characterize delayed rewards for optimizing a policy gradient objective for the parsing model. An empirical evaluation of our approach shows a significant improvement in the performance of entity and relation linking compared to the state of the art. We also demonstrate that a more accurate parsing component enhances the overall performance of QA systems.