Estimation of Clinical Workload and Patient Activity using Deep Learning and Optical Flow

TL;DR

This paper introduces a contactless method combining thermal imaging, object detection, and optical flow to estimate patient activity and caregiver workload in ICU settings, aiding clinical monitoring.

Contribution

It presents a novel approach integrating deep learning and optical flow for estimating clinical workload and patient activity from thermal videos, which is a new application in this context.

Findings

Effective estimation of patient agitation and sedation levels.

Correlation between estimated caregiver motion and clinical workload.

Validation on over 32,000 video frames from ICU patients.

Abstract

Contactless monitoring using thermal imaging has become increasingly proposed to monitor patient deterioration in hospital, most recently to detect fevers and infections during the COVID-19 pandemic. In this letter, we propose a novel method to estimate patient motion and observe clinical workload using a similar technical setup but combined with open source object detection algorithms (YOLOv4) and optical flow. Patient motion estimation was used to approximate patient agitation and sedation, while worker motion was used as a surrogate for caregiver workload. Performance was illustrated by comparing over 32000 frames from videos of patients recorded in an Intensive Care Unit, to clinical agitation scores recorded by clinical workers.