User perceptions of individually-tailored health information in digital apps: development of a scale

TL;DR

This paper introduces a new scale to measure how users perceive the relevance and usefulness of personalized health information in digital apps.

Contribution

The paper presents the development and validation of the SIT scale for assessing perceived tailoring in digital health interventions.

Findings

The SIT scale showed good internal consistency and acceptable test-retest reliability.

A bifactor model with a general factor and two minor subfactors best fit the data.

SIT scores were related to changes in activation and self-efficacy in disease management.

Abstract

Individually tailored health information is thought to have greater effects on patient behavior than generic advice because it is more personally relevant. Most digital health studies, however, do not actually measure the effect of tailoring on study outcomes. To address this gap, we created the Success in Tailoring (SIT) scale which assesses how users perceive information as relevant, useful, and actionable.

The SIT items were chosen to reflect theoretical work on relevance and elements of the Elaboration Likelihood Model. It was administered to participants in a study of a mobile app providing tailored information about chronic disease self-management to persons 40 years of age and older with low health literacy. Participants responded immediately after completing the study intervention and again three months later. Psychometric analyses focused on the measure's reliability, factor structure, and convergent and divergent validity other measures thought to be related and unrelated to it. We assessed test-retest reliability and factorial invariance over administration, and whether the measure predicted changes in key study outcomes.

Analyses were based on responses from 275 participants. The SIT's internal consistency was good, and test-retest reliability was acceptable. Exploratory factor analysis suggested a single-factor solution, although subsequent confirmatory analyses revealed that a bifactor solution with a robust general factor and two minor subfactors fit the data best. The scale was significantly correlated with measures related to its underlying concept and unrelated to measures not related to it, such as physical and cognitive status. Configural and metric, but not scalar, factorial invariance, were confirmed. SIT scores were related to change in activation and disease-management self-efficacy over the course of the study. Confirmatory bifactor analyses supported treating the SIT as essentially unidimensional, with a single total score providing a reliable and valid index of perceived tailoring.

The SIT gives researchers a straightforward means of capturing whether participants feel information is successfully tailored to them. It may be helpful in explaining how personalization, a key feature of many digital health apps, may be related to outcomes.

Figures