Hand Hygiene Assessment via Joint Step Segmentation and Key Action Scorer

TL;DR

This paper introduces a joint step segmentation and key action scoring framework for detailed hand hygiene assessment, utilizing a multi-stage convolution-transformer network and a new annotated dataset to improve accuracy and supervision.

Contribution

The work presents a novel fine-grained learning framework combining step segmentation and key action scoring, along with a new annotated dataset for hand hygiene assessment.

Findings

The proposed method achieves high accuracy in hand hygiene video assessment.

Multi-stage convolution-transformer network improves segmentation robustness.

The dataset enables better supervision and evaluation of hand hygiene actions.

Abstract

Hand hygiene is a standard six-step hand-washing action proposed by the World Health Organization (WHO). However, there is no good way to supervise medical staff to do hand hygiene, which brings the potential risk of disease spread. Existing action assessment works usually make an overall quality prediction on an entire video. However, the internal structures of hand hygiene action are important in hand hygiene assessment. Therefore, we propose a novel fine-grained learning framework to perform step segmentation and key action scorer in a joint manner for accurate hand hygiene assessment. Existing temporal segmentation methods usually employ multi-stage convolutional network to improve the segmentation robustness, but easily lead to over-segmentation due to the lack of the long-range dependence. To address this issue, we design a multi-stage convolution-transformer network for step segmentation. Based on the observation that each hand-washing step involves several key actions which determine the hand-washing quality, we design a set of key action scorers to evaluate the quality of key actions in each step. In addition, there lacks a unified dataset in hand hygiene assessment. Therefore, under the supervision of medical staff, we contribute a video dataset that contains 300 video sequences with fine-grained annotations. Extensive experiments on the dataset suggest that our method well assesses hand hygiene videos and achieves outstanding performance.