Screen Tracking for Clinical Translation of Live Ultrasound Image Analysis Methods

TL;DR

This paper introduces a camera-based screen tracking method to capture ultrasound images for clinical analysis, enabling real-time data overlay and potential AR integration without modifying existing ultrasound systems.

Contribution

A novel, non-intrusive framework for capturing and processing ultrasound images via screen tracking, facilitating clinical translation and augmented reality applications.

Findings

Capture time of approximately 87.66 ms

No physical modification needed for ultrasound devices

Potential for real-time AR overlay in clinical settings

Abstract

Ultrasound (US) imaging is one of the most commonly used non-invasive imaging techniques. However, US image acquisition requires simultaneous guidance of the transducer and interpretation of images, which is a highly challenging task that requires years of training. Despite many recent developments in intra-examination US image analysis, the results are not easy to translate to a clinical setting. We propose a generic framework to extract the US images and superimpose the results of an analysis task, without any need for physical connection or alteration to the US system. The proposed method captures the US image by tracking the screen with a camera fixed at the sonographer's view point and reformats the captured image to the right aspect ratio, in 87.66 +- 3.73ms on average. It is hypothesized that this would enable to input such retrieved image into an image processing pipeline to extract information that can help improve the examination. This information could eventually be projected back to the sonographer's field of view in real time using, for example, an augmented reality (AR) headset.