Unconstrained Face-Mask & Face-Hand Datasets: Building a Computer Vision System to Help Prevent the Transmission of COVID-19

TL;DR

This paper presents a computer vision system that detects face masks, face-hand interactions, and social distancing to help prevent COVID-19 transmission, validated on newly collected and existing datasets with high accuracy.

Contribution

The authors created new real-world datasets for face mask and face-hand interaction detection and developed a system with high generalization for COVID-19 prevention tasks.

Findings

High performance in face mask detection

Effective face-hand interaction detection

Accurate social distance measurement

Abstract

Health organizations advise social distancing, wearing face mask, and avoiding touching face to prevent the spread of coronavirus. Based on these protective measures, we developed a computer vision system to help prevent the transmission of COVID-19. Specifically, the developed system performs face mask detection, face-hand interaction detection, and measures social distance. To train and evaluate the developed system, we collected and annotated images that represent face mask usage and face-hand interaction in the real world. Besides assessing the performance of the developed system on our own datasets, we also tested it on existing datasets in the literature without performing any adaptation on them. In addition, we proposed a module to track social distance between people. Experimental results indicate that our datasets represent the real-world's diversity well. The proposed system achieved very high performance and generalization capacity for face mask usage detection, face-hand interaction detection, and measuring social distance in a real-world scenario on unseen data. The datasets will be available at https://github.com/iremeyiokur/COVID-19-Preventions-Control-System.