VisionCAD: An Integration-Free Radiology Copilot Framework

TL;DR

VisionCAD is a novel framework that enables AI-assisted radiology diagnosis by capturing medical images directly from display screens with a camera, eliminating the need for infrastructure integration.

Contribution

It introduces a camera-based, integration-free system for medical image analysis that maintains diagnostic accuracy comparable to traditional digital image processing.

Findings

Achieves less than 2% F1-score degradation in classification tasks.

Maintains report generation quality within 1% of original images.

Operates with standard hardware and no infrastructure modifications.

Abstract

Widespread clinical deployment of computer-aided diagnosis (CAD) systems is hindered by the challenge of integrating with existing hospital IT infrastructure. Here, we introduce VisionCAD, a vision-based radiological assistance framework that circumvents this barrier by capturing medical images directly from displays using a camera system. The framework operates through an automated pipeline that detects, restores, and analyzes on-screen medical images, transforming camera-captured visual data into diagnostic-quality images suitable for automated analysis and report generation. We validated VisionCAD across diverse medical imaging datasets, demonstrating that our modular architecture can flexibly utilize state-of-the-art diagnostic models for specific tasks. The system achieves diagnostic performance comparable to conventional CAD systems operating on original digital images, with an F1-score degradation typically less than 2\% across classification tasks, while natural language generation metrics for automated reports remain within 1\% of those derived from original images. By requiring only a camera device and standard computing resources, VisionCAD offers an accessible approach for AI-assisted diagnosis, enabling the deployment of diagnostic capabilities in diverse clinical settings without modifications to existing infrastructure.