Generative Context Pair Selection for Multi-hop Question Answering

TL;DR

This paper introduces a generative context selection approach for multi-hop question answering that improves reasoning robustness and outperforms baselines on adversarial datasets by modeling question generation from context pairs.

Contribution

It proposes a novel generative model for context selection in multi-hop QA, enhancing reasoning robustness and addressing dataset biases.

Findings

Achieves 4.9% higher performance on adversarial datasets.

Comparable overall answering performance to state-of-the-art.

Improves model robustness against dataset biases.

Abstract

Compositional reasoning tasks like multi-hop question answering, require making latent decisions to get the final answer, given a question. However, crowdsourced datasets often capture only a slice of the underlying task distribution, which can induce unanticipated biases in models performing compositional reasoning. Furthermore, discriminatively trained models exploit such biases to get a better held-out performance, without learning the right way to reason, as they do not necessitate paying attention to the question representation (conditioning variable) in its entirety, to estimate the answer likelihood. In this work, we propose a generative context selection model for multi-hop question answering that reasons about how the given question could have been generated given a context pair. While being comparable to the state-of-the-art answering performance, our proposed generative passage selection model has a better performance (4.9% higher than baseline) on adversarial held-out set which tests robustness of model's multi-hop reasoning capabilities.