An Investigation of the Test-Retest Reliability of the miniPXI

TL;DR

This study evaluates the test-retest reliability of the miniPXI, a brief player experience measure, comparing it with other established tools across four games and 100 participants, revealing mixed reliability outcomes.

Contribution

It provides an empirical assessment of miniPXI's reliability over time, highlighting its strengths and limitations compared to multi-item and single-item measures.

Findings

miniPXI shows variable test-retest reliability across constructs

Multi-item measures have moderate to good reliability

Single-item indicators like NPS show good reliability

Abstract

Repeated measurements of player experience are crucial in games user research, assessing how different designs evolve over time. However, this necessitates lightweight measurement instruments that are fit for the purpose. In this study, we conduct an examination of the test-retest reliability of the \emph{miniPXI} -- a short variant of the \emph{Player Experience Inventory} (\emph{PXI}), an established measure for measuring player experience. We analyzed test-retest reliability by leveraging four games involving 100 participants, comparing it with four established multi-item measures and single-item indicators such as the Net Promoter Score (\emph{NPS}) and overall enjoyment. The findings show mixed outcomes; the \emph{miniPXI} demonstrated varying levels of test-retest reliability. Some constructs showed good to moderate reliability, while others were less consistent. On the other hand, multi-item measures exhibited moderate to good test-retest reliability, demonstrating their effectiveness in measuring player experiences over time. Additionally, the employed single-item indicators (\emph{NPS} and overall enjoyment) demonstrated good reliability. The results of our study highlight the complexity of player experience evaluations over time, utilizing single and multiple items per construct measures. We conclude that single-item measures may not be appropriate for long-term investigations of more complex PX dimensions and provide practical considerations for the applicability of such measures in repeated measurements.