An Efficient Interaction Human-AI Synergy System Bridging Visual Awareness and Large Language Model for Intensive Care Units

TL;DR

This paper presents a cloud-edge-end human-AI system that enhances ICU data management by integrating visual data extraction and LLM-powered semantic interaction, improving safety and efficiency.

Contribution

It introduces a novel multi-layered architecture combining visual monitoring and language models for real-time, intuitive ICU data interaction.

Findings

Reduces manual data entry errors in ICU monitoring.

Enables voice-based queries for complex patient data.

Improves operational efficiency and safety in ICU environments.

Abstract

Intensive Care Units (ICUs) are critical environments characterized by high-stakes monitoring and complex data management. However, current practices often rely on manual data transcription and fragmented information systems, introducing potential risks to patient safety and operational efficiency. To address these issues, we propose a human-AI synergy system based on a cloud-edge-end architecture, which integrates visual-aware data extraction and semantic interaction mechanisms. Specifically, a visual-aware edge module non-invasively captures real-time physiological data from bedside monitors, reducing manual entry errors. To improve accessibility to fragmented data sources, a semantic interaction module, powered by a Large Language Model (LLM), enables physicians to perform efficient and intuitive voice-based queries over structured patient data. The hierarchical cloud-edge-end deployment ensures low-latency communication and scalable system performance. Our system reduces the cognitive burden on ICU nurses and physicians and demonstrates promising potential for broader applications in intelligent healthcare systems.