A Qualitative Post-Experience Method for Evaluating Changes in VR Presence Experience Over Time

TL;DR

This paper introduces a practical, paper-based post-experience method for evaluating changes in presence over time in VR, effectively detecting break in presence events and enabling objective analysis.

Contribution

It presents a novel, easy-to-apply drawing-based approach for assessing temporal presence variations, including a descriptive model for objective evaluation.

Findings

Method detects all break in presence events

Drawings are highly consistent across users

Parameters correlate with presence intensity

Abstract

A particular measure to evaluate a head-mounted display (HMD) based experience is the state of feeling present in virtual reality. Interruptions of a presence experience - break in presence (BIP) - appearing over time, need to be detected to assess and improve an application. Existing methods either lack in taking these BIPs into account - questionnaires - or are complex in their application and evaluation - physiological and behavioral measures -. To provide a practical approach, we propose a post-experience method in which the users reflect on their experience by drawing a line, indicating their experienced state of presence, in a paper-based drawing template. The amplitude of the drawn line represents the variation of their presence experience over time. We propose a descriptive model that describes temporal variations in the drawings by the definition of relevant points over time - e.g., putting on the HMD -, phases of the experience - e.g., transition into VR - and parameters - e.g., the transition time -. The descriptive model enables us to objectively evaluate user drawings and represent the course of the drawings by a defined set of parameters. An exploratory user study (N=30) showed that the drawings are very consistent, the method can detect all BIPs and shows good indications for representing the intensity of a BIP. With our method practitioners and researchers can accelerate the evaluation and optimization of experiences by evaluating BIPs. The possibility to store objective parameters paves the way for automated evaluation methods and big data approaches.