VRScout: Towards Real-Time, Autonomous Testing of Virtual Reality Games

TL;DR

VRScout is a deep learning agent that autonomously tests VR games in real-time, learning from human demonstrations to ensure quality and safety across diverse environments with minimal training.

Contribution

The paper introduces VRScout, a novel deep learning framework with an enhanced Action Chunking Transformer and adaptive temporal context for autonomous VR game testing.

Findings

Achieves expert-level performance on commercial VR titles

Maintains real-time inference at 60 FPS on consumer hardware

Requires limited training data for effective testing

Abstract

Virtual Reality (VR) has rapidly become a mainstream platform for gaming and interactive experiences, yet ensuring the quality, safety, and appropriateness of VR content remains a pressing challenge. Traditional human-based quality assurance is labor-intensive and cannot scale with the industry's rapid growth. While automated testing has been applied to traditional 2D and 3D games, extending it to VR introduces unique difficulties due to high-dimensional sensory inputs and strict real-time performance requirements. We present VRScout, a deep learning-based agent capable of autonomously navigating VR environments and interacting with virtual objects in a human-like and real-time manner. VRScout learns from human demonstrations using an enhanced Action Chunking Transformer that predicts multi-step action sequences. This enables our agent to capture higher-level strategies and generalize across diverse environments. To balance responsiveness and precision, we introduce a dynamically adjustable sliding horizon that adapts the agent's temporal context at runtime. We evaluate VRScout on commercial VR titles and show that it achieves expert-level performance with only limited training data, while maintaining real-time inference at 60 FPS on consumer-grade hardware. These results position VRScout as a practical and scalable framework for automated VR game testing, with direct applications in both quality assurance and safety auditing.