Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluation

TL;DR

Doctor XAvIer is an explainable AI system that uses BERT to diagnose from doctor-patient dialogues, introducing new evaluation metrics for feature attribution and demonstrating superior performance in clinical data extraction and diagnosis.

Contribution

The paper presents a novel BERT-based diagnostic system with a new feature attribution evaluation metric, advancing explainability in clinical dialogue analysis.

Findings

Integrated gradients outperform Shapley values in explanation quality.

Doctor XAvIer achieves high F1-scores: 0.97 in named entity recognition, 0.91 in diagnosis.

New evaluation metric (FAD curve and N-AUC) for feature attribution methods.

Abstract

We introduce Doctor XAvIer, a BERT-based diagnostic system that extracts relevant clinical data from transcribed patient-doctor dialogues and explains predictions using feature attribution methods. We present a novel performance plot and evaluation metric for feature attribution methods: Feature Attribution Dropping (FAD) curve and its Normalized Area Under the Curve (N-AUC). FAD curve analysis shows that integrated gradients outperforms Shapley values in explaining diagnosis classification. Doctor XAvIer outperforms the baseline with 0.97 F1-score in named entity recognition and symptom pertinence classification and 0.91 F1-score in diagnosis classification.