Generating Realistic Synthetic Head Rotation Data for Extended Reality using Deep Learning

TL;DR

This paper introduces a deep learning-based method to generate realistic synthetic head rotation data for extended reality, addressing data scarcity for training predictive models and enhancing immersive user experiences.

Contribution

It presents a novel application of TimeGAN to generate synthetic head rotation time series, improving data availability for XR systems.

Findings

Synthetic data closely matches real head rotation distributions

The method effectively extends limited datasets

Enhances the training of predictive head motion models

Abstract

Extended Reality is a revolutionary method of delivering multimedia content to users. A large contributor to its popularity is the sense of immersion and interactivity enabled by having real-world motion reflected in the virtual experience accurately and immediately. This user motion, mainly caused by head rotations, induces several technical challenges. For instance, which content is generated and transmitted depends heavily on where the user is looking. Seamless systems, taking user motion into account proactively, will therefore require accurate predictions of upcoming rotations. Training and evaluating such predictors requires vast amounts of orientational input data, which is expensive to gather, as it requires human test subjects. A more feasible approach is to gather a modest dataset through test subjects, and then extend it to a more sizeable set using synthetic data generation methods. In this work, we present a head rotation time series generator based on TimeGAN, an extension of the well-known Generative Adversarial Network, designed specifically for generating time series. This approach is able to extend a dataset of head rotations with new samples closely matching the distribution of the measured time series.