MaskBot: Real-time Robotic Projection Mapping with Head Motion Tracking

TL;DR

MaskBot is a novel real-time robotic projection mapping system that uses a 6-DoF robot and face tracking to enhance projection area and reduce latency for applications like face modification and adversarial image generation.

Contribution

It introduces the first robotic projection mapping system that dynamically adjusts to user movement, expanding projection capabilities and reducing latency compared to existing methods.

Findings

Increased projection area on the face.

Reduced latency through robotic positioning.

Potential for generating adversarial face images.

Abstract

The projection mapping systems on the human face is limited by the latency and the movement of the users. The area of the projection is restricted by the position of the projectors and the cameras. We are introducing MaskBot, a real-time projection mapping system operated by a 6 Degrees of Freedom (DoF) collaborative robot. The collaborative robot locates the projector and camera in normal position to the face of the user to increase the projection area and to reduce the latency of the system. A webcam is used to detect the face and to sense the robot-user distance to modify the projection size and orientation. MaskBot projects different images on the face of the user, such as face modifications, make-up, and logos. In contrast to the existing methods, the presented system is the first that introduces a robotic projection mapping. One of the prospective applications is to acquire a dataset of adversarial images to challenge face detection DNN systems, such as Face ID.