VR.net: A Real-world Dataset for Virtual Reality Motion Sickness Research

TL;DR

VR.net is a comprehensive, accurately labeled dataset of real-world VR gameplay videos with motion sickness-related annotations, enabling improved machine learning research for motion sickness detection and prediction.

Contribution

The paper introduces VR.net, a large-scale, automatically labeled dataset from real VR gameplay, addressing the lack of diverse, accurately annotated data for motion sickness research.

Findings

VR.net contains approximately 12 hours of gameplay videos across 10 genres.

The dataset includes detailed motion sickness-related labels for each frame.

Initial applications demonstrate VR.net's utility in risk detection and sickness level prediction.

Abstract

Researchers have used machine learning approaches to identify motion sickness in VR experience. These approaches demand an accurately-labeled, real-world, and diverse dataset for high accuracy and generalizability. As a starting point to address this need, we introduce `VR.net', a dataset offering approximately 12-hour gameplay videos from ten real-world games in 10 diverse genres. For each video frame, a rich set of motion sickness-related labels, such as camera/object movement, depth field, and motion flow, are accurately assigned. Building such a dataset is challenging since manual labeling would require an infeasible amount of time. Instead, we utilize a tool to automatically and precisely extract ground truth data from 3D engines' rendering pipelines without accessing VR games' source code. We illustrate the utility of VR.net through several applications, such as risk factor detection and sickness level prediction. We continuously expand VR.net and envision its next version offering 10X more data than the current form. We believe that the scale, accuracy, and diversity of VR.net can offer unparalleled opportunities for VR motion sickness research and beyond.