Fully Automated Hand Hygiene Monitoring\\in Operating Room using 3D Convolutional Neural Network

TL;DR

This paper presents a fully automated system using 3D CNNs to monitor hand hygiene compliance of anesthesiologists in operating rooms, aiming to reduce hospital-acquired infections through video analysis.

Contribution

It introduces a novel application of 3D CNNs for real-time hand hygiene detection in OR videos, leveraging transfer learning and focusing on alcohol-based hand rubbing actions.

Findings

Achieved 76% accuracy in classifying hand rubbing actions.

Transfer learning from Kinetics-400 improved model performance.

Optical flow was not beneficial for this dataset.

Abstract

Hand hygiene is one of the most significant factors in preventing hospital acquired infections (HAI) which often be transmitted by medical staffs in contact with patients in the operating room (OR). Hand hygiene monitoring could be important to investigate and reduce the outbreak of infections within the OR. However, an effective monitoring tool for hand hygiene compliance is difficult to develop due to the visual complexity of the OR scene. Recent progress in video understanding with convolutional neural net (CNN) has increased the application of recognition and detection of human actions. Leveraging this progress, we proposed a fully automated hand hygiene monitoring tool of the alcohol-based hand rubbing action of anesthesiologists on OR video using spatio-temporal features with 3D CNN. First, the region of interest (ROI) of anesthesiologists' upper body were detected and cropped. A temporal smoothing filter was applied to the ROIs. Then, the ROIs were given to a 3D CNN and classified into two classes: rubbing hands or other actions. We observed that a transfer learning from Kinetics-400 is beneficial and the optical flow stream was not helpful in our dataset. The final accuracy, precision, recall and F1 score in testing is 0.76, 0.85, 0.65 and 0.74, respectively.