Smooth head tracking for virtual reality applications

TL;DR

This paper introduces a novel RGBD-based head-tracking method for virtual reality that provides smooth, markerless, and ergonomic camera control by predicting and interpolating head movements, enhancing user experience in virtual environments.

Contribution

The work presents a new real-time head-tracking approach using facial features and Pythagorean Hodograph interpolation for smooth, continuous, and markerless head motion modeling in VR.

Findings

Provides smooth, jerk-free head trajectories

Adapts to any rendering frame rate

Eliminates need for tracking markers

Abstract

In this work, we propose a new head-tracking solution for human-machine real-time interaction with virtual 3D environments. This solution leverages RGBD data to compute virtual camera pose according to the movements of the user's head. The process starts with the extraction of a set of facial features from the images delivered by the sensor. Such features are matched against their respective counterparts in a reference image for the computation of the current head pose. Afterwards, a prediction approach is used to guess the most likely next head move (final pose). Pythagorean Hodograph interpolation is then adapted to determine the path and local frames taken between the two poses. The result is a smooth head trajectory that serves as an input to set the camera in virtual scenes according to the user's gaze. The resulting motion model has the advantage of being: continuous in time, it adapts to any frame rate of rendering; it is ergonomic, as it frees the user from wearing tracking markers; it is smooth and free from rendering jerks; and it is also torsion and curvature minimizing as it produces a path with minimum bending energy.