Automated Interpretable 2D Video Extraction from 3D Echocardiography

TL;DR

This paper introduces an automated deep learning-based method to extract standard 2D echocardiography views from 3D ultrasound data, enabling clinicians to interpret 3D scans in familiar formats with high accuracy.

Contribution

It presents a novel automated approach combining deep learning and heuristics to reconstruct standard 2D views from 3D echocardiography volumes, validated across multiple hospitals.

Findings

96% accuracy in view classification on 1,600 videos

Effective detection of cardiac abnormalities using AI models

Preservation of spatial and diagnostic features in extracted videos

Abstract

Although the heart has complex three-dimensional (3D) anatomy, conventional medical imaging with cardiac ultrasound relies on a series of 2D videos showing individual cardiac structures. 3D echocardiography is a developing modality that now offers adequate image quality for clinical use, with potential to streamline acquisition and improve assessment of off-axis features. We propose an automated method to select standard 2D views from 3D cardiac ultrasound volumes, allowing physicians to interpret the data in their usual format while benefiting from the speed and usability of 3D scanning. Applying a deep learning view classifier and downstream heuristics based on anatomical landmarks together with heuristics provided by cardiologists, we reconstruct standard echocardiography views. This approach was validated by three cardiologists in blinded evaluation (96\% accuracy in 1,600 videos from 2 hospitals). The downstream 2D videos were also validated in their ability to detect cardiac abnormalities using AI echocardiography models (EchoPrime and PanEcho) as well as ability to generate clinical-grade measurements of cardiac anatomy (EchoNet-Measurement). We demonstrated that the extracted 2D videos preserve spatial calibration and diagnostic features, allowing clinicians to obtain accurate real-world interpretations from 3D volumes. We release the code and a dataset of 29 3D echocardiography videos https://github.com/echonet/3d-echo .