Red Dragon AI at TextGraphs 2021 Shared Task: Multi-Hop Inference Explanation Regeneration by Matching Expert Ratings

TL;DR

This paper presents a system for multi-hop inference explanation regeneration in science question answering, leveraging expert relevance ratings, retrieval, language models, and ensembling, achieving second place in the TextGraphs 2021 Shared Task.

Contribution

It introduces a novel approach that incorporates expert relevance ratings into multi-hop inference explanation generation, improving over previous methods.

Findings

Achieved second place in the Shared Task leaderboard.

Effectively combined retrieval, relevance prediction, and ensembling.

Enhanced explanation quality with expert-rated relevance data.

Abstract

Creating explanations for answers to science questions is a challenging task that requires multi-hop inference over a large set of fact sentences. This year, to refocus the Textgraphs Shared Task on the problem of gathering relevant statements (rather than solely finding a single 'correct path'), the WorldTree dataset was augmented with expert ratings of 'relevance' of statements to each overall explanation. Our system, which achieved second place on the Shared Task leaderboard, combines initial statement retrieval; language models trained to predict the relevance scores; and ensembling of a number of the resulting rankings. Our code implementation is made available at https://github.com/mdda/worldtree_corpus/tree/textgraphs_2021