Methodological Explainability Evaluation of an Interpretable Deep Learning Model for Post-Hepatectomy Liver Failure Prediction Incorporating Counterfactual Explanations and Layerwise Relevance Propagation: A Prospective In Silico Trial

TL;DR

This study develops and evaluates an interpretable AI model for predicting post-hepatectomy liver failure, integrating counterfactual explanations and LRP, and demonstrates improved clinician decision-making through a comprehensive in silico trial.

Contribution

Introduces a novel explainability framework for AI in medical prediction, combining counterfactuals, LRP, and a methodological evaluation approach.

Findings

Model explanations aligned with known biomarkers.

High usability of the AI system for clinicians.

Clinicians' prediction accuracy and confidence improved with explanations.

Abstract

Artificial intelligence (AI)-based decision support systems have demonstrated value in predicting post-hepatectomy liver failure (PHLF) in hepatocellular carcinoma (HCC). However, they often lack transparency, and the impact of model explanations on clinicians' decisions has not been thoroughly evaluated. Building on prior research, we developed a variational autoencoder-multilayer perceptron (VAE-MLP) model for preoperative PHLF prediction. This model integrated counterfactuals and layerwise relevance propagation (LRP) to provide insights into its decision-making mechanism. Additionally, we proposed a methodological framework for evaluating the explainability of AI systems. This framework includes qualitative and quantitative assessments of explanations against recognized biomarkers, usability evaluations, and an in silico clinical trial. Our evaluations demonstrated that the model's explanation correlated with established biomarkers and exhibited high usability at both the case and system levels. Furthermore, results from the three-track in silico clinical trial showed that clinicians' prediction accuracy and confidence increased when AI explanations were provided.