Factored Attention and Embedding for Unstructured-view Topic-related Ultrasound Report Generation

TL;DR

This paper introduces FAE-Gen, a novel model for automated ultrasound report generation that effectively handles unstructured views and topic-related descriptions, improving accuracy and relevance in clinical cardiovascular ultrasound reports.

Contribution

The paper presents a new factored attention and embedding model specifically designed for unstructured-view, topic-related ultrasound report generation, addressing limitations of previous methods.

Findings

FAE-Gen outperforms seven baseline metrics in evaluations.

The model effectively captures morphological features across views.

Qualitative analysis confirms the model's ability to generate relevant reports.

Abstract

Echocardiography is widely used to clinical practice for diagnosis and treatment, e.g., on the common congenital heart defects. The traditional manual manipulation is error-prone due to the staff shortage, excess workload, and less experience, leading to the urgent requirement of an automated computer-aided reporting system to lighten the workload of ultrasonologists considerably and assist them in decision making. Despite some recent successful attempts in automatical medical report generation, they are trapped in the ultrasound report generation, which involves unstructured-view images and topic-related descriptions. To this end, we investigate the task of the unstructured-view topic-related ultrasound report generation, and propose a novel factored attention and embedding model (termed FAE-Gen). The proposed FAE-Gen mainly consists of two modules, i.e., view-guided factored attention and topic-oriented factored embedding, which 1) capture the homogeneous and heterogeneous morphological characteristic across different views, and 2) generate the descriptions with different syntactic patterns and different emphatic contents for different topics. Experimental evaluations are conducted on a to-be-released large-scale clinical cardiovascular ultrasound dataset (CardUltData). Both quantitative comparisons and qualitative analysis demonstrate the effectiveness and the superiority of FAE-Gen over seven commonly-used metrics.