Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

TL;DR

The BioASQ 2021 challenge focused on advancing biomedical semantic indexing and question answering, introducing a new COVID-19 related task, with 42 teams participating and showing continuous performance improvements.

Contribution

This paper provides an overview of the ninth BioASQ challenge, including the new Synergy question answering task for COVID-19, and reports on the participation and performance results.

Findings

Performance improvements over baselines

Introduction of COVID-19 specific question answering task

High participation with 42 teams and 170 systems

Abstract

Advancing the state-of-the-art in large-scale biomedical semantic indexing and question answering is the main focus of the BioASQ challenge. BioASQ organizes respective tasks where different teams develop systems that are evaluated on the same benchmark datasets that represent the real information needs of experts in the biomedical domain. This paper presents an overview of the ninth edition of the BioASQ challenge in the context of the Conference and Labs of the Evaluation Forum (CLEF) 2021. In this year, a new question answering task, named Synergy, is introduced to support researchers studying the COVID-19 disease and measure the ability of the participating teams to discern information while the problem is still developing. In total, 42 teams with more than 170 systems were registered to participate in the four tasks of the challenge. The evaluation results, similarly to previous years, show a performance gain against the baselines which indicates the continuous improvement of the state-of-the-art in this field.