L2R2: Leveraging Ranking for Abductive Reasoning

TL;DR

This paper introduces L2R2, a ranking-based approach for abductive reasoning in natural language inference, utilizing learning-to-rank models with pre-trained language models to improve plausibility assessment.

Contribution

It proposes a novel ranking perspective for $ ext{ extalpha}$NLI, reformulating hypothesis evaluation as a ranking problem and applying learning-to-rank methods with pre-trained models.

Findings

Achieves state-of-the-art results on ART dataset

Reveals ranking-based approach improves plausibility discrimination

Demonstrates effectiveness of learning-to-rank in abductive reasoning

Abstract

The abductive natural language inference task ($\alpha$NLI) is proposed to evaluate the abductive reasoning ability of a learning system. In the $\alpha$NLI task, two observations are given and the most plausible hypothesis is asked to pick out from the candidates. Existing methods simply formulate it as a classification problem, thus a cross-entropy log-loss objective is used during training. However, discriminating true from false does not measure the plausibility of a hypothesis, for all the hypotheses have a chance to happen, only the probabilities are different. To fill this gap, we switch to a ranking perspective that sorts the hypotheses in order of their plausibilities. With this new perspective, a novel $L2R^2$ approach is proposed under the learning-to-rank framework. Firstly, training samples are reorganized into a ranking form, where two observations and their hypotheses are treated as the query and a set of candidate documents respectively. Then, an ESIM model or pre-trained language model, e.g. BERT or RoBERTa, is obtained as the scoring function. Finally, the loss functions for the ranking task can be either pair-wise or list-wise for training. The experimental results on the ART dataset reach the state-of-the-art in the public leaderboard.