How Do We Evaluate Experiences in Immersive Environments?

TL;DR

This paper maps current evaluation practices in immersive environments, highlighting their diversity and proposing a unified, user-centered, and sustainable approach to improve comparability and progress in the field.

Contribution

It provides a comprehensive scoping review of 375 papers, revealing the fragmented landscape and suggesting integrated, refined measures and open practices for future immersive experience evaluation.

Findings

Evaluation practices are domain-specific and heterogeneous.

Researchers should integrate and refine existing instruments.

Open, sustainable evaluation practices are essential for progress.

Abstract

How do we evaluate experiences in immersive environments? Despite decades of research in immersive technologies such as virtual reality, the field remains fragmented. Studies rely on overlapping constructs, heterogeneous instruments, and little agreement on what counts as immersive experience. To better understand this landscape, we conducted a bottom-up scoping review of 375 papers published in ACM CHI, UIST, VRST, SUI, IEEE VR, ISMAR, and TVCG. Our analysis reveals that evaluation practices are often domain- and purpose-specific, shaped more by local choices than by shared standards. Yet this diversity also points to new directions. Instead of multiplying instruments, researchers benefit from integrating and refining them into smarter measures. Rather than focusing only on system outputs, evaluations must center the user's lived experience. Computational modeling offers opportunities to bridge signals across methods, but lasting progress requires open and sustainable evaluation practices that support comparability and reuse. Ultimately, our contribution is to map current practices and outline a forward-looking agenda for immersive experience research.