CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures

TL;DR

This paper introduces the Multilingual CasiMedicos-Arg dataset, a novel resource for medical question answering that includes doctor-written explanations annotated with argumentative structures across four languages.

Contribution

It presents the first multilingual dataset with annotated explanations and argument structures for medical diagnosis, aiding AI and education in explainability.

Findings

Baseline models perform competitively on argument mining tasks.

The dataset covers four languages with extensive annotations.

It supports research in explainable AI for medicine.

Abstract

Explaining Artificial Intelligence (AI) decisions is a major challenge nowadays in AI, in particular when applied to sensitive scenarios like medicine and law. However, the need to explain the rationale behind decisions is a main issue also for human-based deliberation as it is important to justify \textit{why} a certain decision has been taken. Resident medical doctors for instance are required not only to provide a (possibly correct) diagnosis, but also to explain how they reached a certain conclusion. Developing new tools to aid residents to train their explanation skills is therefore a central objective of AI in education. In this paper, we follow this direction, and we present, to the best of our knowledge, the first multilingual dataset for Medical Question Answering where correct and incorrect diagnoses for a clinical case are enriched with a natural language explanation written by doctors. These explanations have been manually annotated with argument components (i.e., premise, claim) and argument relations (i.e., attack, support), resulting in the Multilingual CasiMedicos-Arg dataset which consists of 558 clinical cases in four languages (English, Spanish, French, Italian) with explanations, where we annotated 5021 claims, 2313 premises, 2431 support relations, and 1106 attack relations. We conclude by showing how competitive baselines perform over this challenging dataset for the argument mining task.