Chain of Condition: Construct, Verify and Solve Conditions for Conditional Question Answering

TL;DR

This paper introduces a novel prompting method called Chain of Condition for conditional question answering, which explicitly constructs and verifies logical conditions to improve answer accuracy and detect missing information, outperforming existing models.

Contribution

The paper presents a new approach that explicitly constructs, verifies, and solves logical conditions in CQA, achieving state-of-the-art results with minimal supervision.

Findings

Outperforms existing prompting baselines on benchmark datasets.

Enables large language models to surpass supervised models with few examples.

Establishes a new state-of-the-art in conditional question answering.

Abstract

Conditional question answering (CQA) is an important task that aims to find probable answers and identify missing conditions. Existing approaches struggle with CQA due to two challenges: (1) precisely identifying necessary conditions and the logical relationship, and (2) verifying conditions to detect any that are missing. In this paper, we propose a novel prompting approach, Chain of condition, by first identifying all conditions and constructing their logical relationships explicitly according to the document, then verifying whether these conditions are satisfied, finally solving the logical expression to indicate any missing conditions and generating the answer accordingly. Experiments on two CQA benchmark datasets show our chain of condition outperforms existing prompting baselines, establishing a new state of the art. Furthermore, with only a few examples, our method can facilitate GPT-3.5-Turbo or GPT-4 to outperform all existing supervised models.