Intensive, Repeated Self-Report Measures: Should We Be Concerned About Changes in Data Quality Over Time?

TL;DR

This paper explores how repeated self-report measures in mobile health apps may affect data quality over time.

Contribution

The paper identifies four phenomena that may cause noninvariance in repeated self-report data collection.

Findings

The time required to complete assessments may change over time.

The rate of missing data and careless responding can increase with repeated use.

Reactivity components may influence responses as assessments are repeated.

Abstract

Intensive, repeated self-report measures are an important tool for behavioral and medical researchers and practitioners who are concerned with the dynamic interplay among variables at a granular level. Many mobile health applications rely on accurate measurement of immediate states and environments for both assessment and intervention delivery. Techniques for capturing repeated momentary assessments yield data with several salutary qualities: recall bias is minimized relative to assessments that rely on much longer recall periods; measurements are taken in individuals’ everyday environments; and dense, repeated measures allow a new window into the processes transpiring between individuals and their environments. In this paper, we highlight several features of repeatedly completing momentary assessments that may change the nature or quality of the data collected over time. Several lines of inquiry are discussed that call into question the presumption that there is invariance in how people complete repeated assessments over time. A result of this possibility could be a reduction in data quality. We present 4 phenomena, with selected results, that may induce noninvariance in repeated measures: the amount of time required to complete assessments, the rate of missing data, the degree of careless responding, and the presence of several components of reactivity. In each of these areas, we found evidence that changes could occur over time, and we consider how data might be affected by such changes. Our conclusion is that researchers should be aware that changes can occur over time and that these changes may affect data quality.

Figures