Multi-modality Regional Alignment Network for Covid X-Ray Survival Prediction and Report Generation
Zhusi Zhong, Jie Li, John Sollee, Scott Collins, Harrison Bai, Paul, Zhang, Terrence Healey, Michael Atalay, Xinbo Gao, Zhicheng Jiao

TL;DR
This paper introduces MRANet, an explainable multi-modality model that improves COVID-19 X-ray survival prediction and radiology report generation by focusing on high-risk regions and enhancing interpretability.
Contribution
The study presents a novel multi-modality regional alignment network with a survival attention mechanism and cross-LLMs alignment for improved medical report generation and prognosis prediction.
Findings
Enhanced survival prediction accuracy on multi-center datasets.
Generated reports with higher clinical relevance and explainability.
Demonstrated robustness and effectiveness of each module within MRANet.
Abstract
In response to the worldwide COVID-19 pandemic, advanced automated technologies have emerged as valuable tools to aid healthcare professionals in managing an increased workload by improving radiology report generation and prognostic analysis. This study proposes Multi-modality Regional Alignment Network (MRANet), an explainable model for radiology report generation and survival prediction that focuses on high-risk regions. By learning spatial correlation in the detector, MRANet visually grounds region-specific descriptions, providing robust anatomical regions with a completion strategy. The visual features of each region are embedded using a novel survival attention mechanism, offering spatially and risk-aware features for sentence encoding while maintaining global coherence across tasks. A cross LLMs alignment is employed to enhance the image-to-text transfer process, resulting in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCOVID-19 diagnosis using AI · Radiomics and Machine Learning in Medical Imaging
