Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop Alignment

TL;DR

This paper introduces a rapid, markerless projection mapping technique that uses a closed-loop system with a DMD projector and camera to track and project onto moving surfaces without prior calibration, achieving real-time performance.

Contribution

It presents a novel fast projection mapping method using a fiducial pattern and closed-loop alignment, enabling real-time, markerless projection onto moving textured surfaces.

Findings

Runs at 400 fps on CPU

Enables arbitrary video projection on textured surfaces

Eliminates need for calibration between camera and projector

Abstract

This paper presents a fast projection mapping method for moving image content projected onto a markerless planar surface using a low-latency Digital Micromirror Device (DMD) projector. By adopting a closed-loop alignment approach, in which not only the surface texture but also the projected image is tracked by a camera, the proposed method is free from a calibration or position adjustment between the camera and projector. We designed fiducial patterns to be inserted into a fast flapping sequence of binary frames of the DMD projector, which allows the simultaneous tracking of the surface texture and a fiducial geometry separate from a single image captured by the camera. The proposed method implemented on a CPU runs at 400 fps and enables arbitrary video contents to be "stuck" onto a variety of textured surfaces.