EchoNarrator: Generating natural text explanations for ejection fraction predictions

TL;DR

EchoNarrator is a model that predicts ejection fraction from cardiac ultrasound data and generates human-like natural language explanations, enhancing interpretability and trust in clinical settings.

Contribution

The paper introduces a novel approach combining EF prediction with natural language explanations in a single model, improving interpretability of deep learning in cardiology.

Findings

Achieves EF prediction accuracy comparable to state-of-the-art methods.

Provides meaningful and accurate natural language explanations for predictions.

Demonstrates the potential for improved trust in AI models for clinical use.

Abstract

Ejection fraction (EF) of the left ventricle (LV) is considered as one of the most important measurements for diagnosing acute heart failure and can be estimated during cardiac ultrasound acquisition. While recent successes in deep learning research successfully estimate EF values, the proposed models often lack an explanation for the prediction. However, providing clear and intuitive explanations for clinical measurement predictions would increase the trust of cardiologists in these models. In this paper, we explore predicting EF measurements with Natural Language Explanation (NLE). We propose a model that in a single forward pass combines estimation of the LV contour over multiple frames, together with a set of modules and routines for computing various motion and shape attributes that are associated with ejection fraction. It then feeds the attributes into a large language model to generate text that helps to explain the network's outcome in a human-like manner. We provide experimental evaluation of our explanatory output, as well as EF prediction, and show that our model can provide EF comparable to state-of-the-art together with meaningful and accurate natural language explanation to the prediction. The project page can be found at https://github.com/guybenyosef/EchoNarrator .