NEURAL: Attention-Guided Pruning for Unified Multimodal Resource-Constrained Clinical Evaluation

TL;DR

NEURAL introduces a semantics-guided, attention-based image pruning framework that compresses medical images into graph representations, maintaining diagnostic accuracy while significantly reducing data size for resource-limited clinical environments.

Contribution

This work presents a novel attention-guided pruning method that creates a unified graph-based representation combining visual and knowledge graphs for efficient medical image analysis.

Findings

Achieves 93.4-97.7% data size reduction with high diagnostic performance (0.88-0.95 AUC).

Outperforms baseline models using uncompressed data.

Enables efficient, task-agnostic clinical workflows and teleradiology.

Abstract

The rapid growth of multimodal medical imaging data presents significant storage and transmission challenges, particularly in resource-constrained clinical settings. We propose NEURAL, a novel framework that addresses this by using semantics-guided data compression. Our approach repurposes cross-attention scores between the image and its radiological report from a fine-tuned generative vision-language model to structurally prune chest X-rays, preserving only diagnostically critical regions. This process transforms the image into a highly compressed, graph representation. This unified graph-based representation fuses the pruned visual graph with a knowledge graph derived from the clinical report, creating a universal data structure that simplifies downstream modeling. Validated on the MIMIC-CXR and CheXpert Plus dataset for pneumonia detection, NEURAL achieves a 93.4-97.7\% reduction in image data size while maintaining a high diagnostic performance of 0.88-0.95 AUC, outperforming other baseline models that use uncompressed data. By creating a persistent, task-agnostic data asset, NEURAL resolves the trade-off between data size and clinical utility, enabling efficient workflows and teleradiology without sacrificing performance. Our NEURAL code is available at https://github.com/basiralab/NEURAL.