System Test Generation for Virtual Reality Applications using Scenario Models

TL;DR

UltraInstinctVR is a new systematic testing approach for VR applications that uses scenario models to automate test generation and execution, improving failure detection and bug identification.

Contribution

The paper introduces UltraInstinctVR, a novel scenario-based testing method specifically designed for VR applications, enhancing testing effectiveness over existing approaches.

Findings

UltraInstinctVR outperforms state-of-the-art VR testing tools in coverage and failure detection.

It detects unique failures and provides insights into real-world bugs in VR applications.

Empirical evaluation on 10 open-source VR apps demonstrates its effectiveness.

Abstract

Virtual Reality (VR) applications are increasingly being integrated across a wide range of domains, including surgical training and industrial marketing. However, the long-term adoption and maintenance of VR applications remain limited, particularly due to the lack of effective, systematic, and reproducible software testing approaches tailored to their unique characteristics. To address this issue, we introduce UltraInstinctVR, a novel testing approach for VR applications. Relying on predefined VR models (scenarios), it automates the generation and execution of concrete VR system tests. In our empirical evaluation, we compare UltraInstinctVR with state-of-the-art automated VR testing approaches in terms of coverage and failure detection on 10 open-source VR applications. The results show that UltraInstinctVR outperforms existing automated tools for detecting unique failures and provides valuable insights for identifying real-world bugs in VR applications.