Science Checker: Extractive-Boolean Question Answering For Scientific Fact Checking

TL;DR

Science Checker is a multi-task system that combines extractive summarization and Boolean question answering to verify scientific facts efficiently from research articles, achieving high accuracy on large medical datasets.

Contribution

It introduces a novel multi-task approach integrating summarization and Boolean QA for scientific fact checking, with a lightweight architecture and strong performance.

Findings

Achieved 4% error rate and 95.6% F1-score.

Validated on 3 million articles from Europe PMC.

Effective across medical and health domains.

Abstract

With the explosive growth of scientific publications, making the synthesis of scientific knowledge and fact checking becomes an increasingly complex task. In this paper, we propose a multi-task approach for verifying the scientific questions based on a joint reasoning from facts and evidence in research articles. We propose an intelligent combination of (1) an automatic information summarization and (2) a Boolean Question Answering which allows to generate an answer to a scientific question from only extracts obtained after summarization. Thus on a given topic, our proposed approach conducts structured content modeling based on paper abstracts to answer a scientific question while highlighting texts from paper that discuss the topic. We based our final system on an end-to-end Extractive Question Answering (EQA) combined with a three outputs classification model to perform in-depth semantic understanding of a question to illustrate the aggregation of multiple responses. With our light and fast proposed architecture, we achieved an average error rate of 4% and a F1-score of 95.6%. Our results are supported via experiments with two QA models (BERT, RoBERTa) over 3 Million Open Access (OA) articles in the medical and health domains on Europe PMC.