The Development and Validation of the Technology-Supported Reflection Inventory

TL;DR

The paper introduces the Technology-Supported Reflection Inventory (TSRI), a validated scale for evaluating how effectively interactive systems support reflection, aiding comparison and development of reflection-supporting artefacts in HCI.

Contribution

It presents the development and validation of a new scale, TSRI, for assessing reflection support in interactive systems, addressing evaluation challenges in HCI.

Findings

TSRI has 9 items after factor analysis

TSRI demonstrates good test-retest validity

TSRI shows strong construct validity

Abstract

Reflection is an often addressed design goal in Human-Computer Interaction (HCI) research. An increasing number of artefacts for reflection have been developed in recent years. However, evaluating if and how an interactive technology helps a user reflect is still complex. This makes it difficult to compare artefacts (or prototypes) for reflection, impeding future design efforts. To address this issue, we developed the \emph{Technology-Supported Reflection Inventory} (TSRI), which is a scale that evaluates how effectively a system supports reflection. We first created a list of possible scale items based on past work in defining reflection. The items were then reviewed by experts. Next, we performed exploratory factor analysis to reduce the scale to its final length of nine items. Subsequently, we confirmed test-retest validity of our instrument, as well as its construct validity. The TSRI enables researchers and practitioners to compare prototypes designed to support reflection.