QMUL-SDS at SCIVER: Step-by-Step Binary Classification for Scientific Claim Verification

TL;DR

This paper presents a step-by-step binary classification approach for scientific claim verification using BioBERT, improving accuracy in the SCIVER shared task by sequentially selecting relevant abstracts, rationales, and labels.

Contribution

The authors introduce a multi-stage classification method with BioBERT for claim verification, enhancing performance over baseline systems in a scientific fact-checking benchmark.

Findings

Achieved substantial improvements over baseline in the SCIVER dev set.

Ranked No. 4 on the leaderboard among participating teams.

Demonstrated effectiveness of step-by-step classification in scientific claim verification.

Abstract

Scientific claim verification is a unique challenge that is attracting increasing interest. The SCIVER shared task offers a benchmark scenario to test and compare claim verification approaches by participating teams and consists in three steps: relevant abstract selection, rationale selection and label prediction. In this paper, we present team QMUL-SDS's participation in the shared task. We propose an approach that performs scientific claim verification by doing binary classifications step-by-step. We trained a BioBERT-large classifier to select abstracts based on pairwise relevance assessments for each <claim, title of the abstract> and continued to train it to select rationales out of each retrieved abstract based on <claim, sentence>. We then propose a two-step setting for label prediction, i.e. first predicting "NOT_ENOUGH_INFO" or "ENOUGH_INFO", then label those marked as "ENOUGH_INFO" as either "SUPPORT" or "CONTRADICT". Compared to the baseline system, we achieve substantial improvements on the dev set. As a result, our team is the No. 4 team on the leaderboard.