A Generate-Validate Approach to Answering Questions about Qualitative Relationships

TL;DR

This paper introduces a generate-validate framework for answering questions about qualitative relationships, improving over traditional semantic parsers by enabling better transfer learning and achieving a 7.93% performance boost.

Contribution

It proposes a novel generate-validate approach that replaces direct semantic parsing, enhancing transfer learning and significantly improving accuracy in qualitative relationship question answering.

Findings

Outperforms state-of-the-art by 7.93%

Enables better transfer learning

Improves accuracy in qualitative relationship questions

Abstract

Qualitative relationships describe how increasing or decreasing one property (e.g. altitude) affects another (e.g. temperature). They are an important aspect of natural language question answering and are crucial for building chatbots or voice agents where one may enquire about qualitative relationships. Recently a dataset about question answering involving qualitative relationships has been proposed, and a few approaches to answer such questions have been explored, in the heart of which lies a semantic parser that converts the natural language input to a suitable logical form. A problem with existing semantic parsers is that they try to directly convert the input sentences to a logical form. Since the output language varies with each application, it forces the semantic parser to learn almost everything from scratch. In this paper, we show that instead of using a semantic parser to produce the logical form, if we apply the generate-validate framework i.e. generate a natural language description of the logical form and validate if the natural language description is followed from the input text, we get a better scope for transfer learning and our method outperforms the state-of-the-art by a large margin of 7.93%.