Some Lessons Learned Running Virtual Reality Experiments Out of the Laboratory

TL;DR

This paper discusses the challenges, opportunities, and lessons learned from transitioning VR experiments from controlled laboratory settings to real-world environments, highlighting methodological adaptations and diverse participant engagement.

Contribution

It provides practical insights and lessons learned on conducting VR experiments outside the lab, including tool improvements and data collection strategies.

Findings

Successful adaptation of VR experiments for in-the-wild settings

Enhanced participant diversity compared to lab-based studies

Identified key challenges and solutions for remote VR experimentation

Abstract

In the past twelve months, our team has had to move rapidly from conducting most of our user experiments in a laboratory setting, to running experiments in the wild away from the laboratory and without direct synchronous oversight from an experimenter. This has challenged us to think about what types of experiment we can run, and to improve our tools and methods to allow us to reliably capture the necessary data. It has also offered us an opportunity to engage with a more diverse population than we would normally engage with in the laboratory. In this position paper we elaborate on the challenges and opportunities, and give some lessons learned from our own experience.