Smartphone App–Based Survey Deployment Patterns and Longitudinal Response Rate: Randomized Controlled Trial

TL;DR

A study found that giving smaller batches of surveys every two weeks instead of all at once every four weeks improved long-term response rates in a smartphone app-based health study.

Contribution

The study introduces a novel survey deployment strategy to combat survey fatigue in longitudinal digital health research.

Findings

The experimental group had higher response rates over time compared to the control group.

Stratified analyses showed younger participants had higher response rates in the experimental group.

The proportion of participants not returning any surveys increased more in the control group.

Abstract

Survey fatigue is a common challenge in longitudinal studies, particularly when using smartphone apps to collect survey data. Evidence-based strategies are needed to maintain longitudinal response rates.

This study aims to evaluate the effect of a more frequent smartphone-administered survey deployment strategy with smaller survey batches on participant response rates over an extended period.

We conducted a randomized controlled trial (NCT04752657) embedded in the electronic Framingham Heart Study cohorts between June 2021 and December 2023. Participants were randomly allocated to receive a full set of surveys every 4 weeks (control group) or half of the survey set biweekly, such that the full set is completed every 4 weeks (experimental group). Randomization was stratified by age (≤75 y vs >75 y) and phone type (Android vs iPhone). Married couples were assigned to the same group using a blocked randomization approach. The primary outcome was the proportion of surveys returned per participant assessed longitudinally across four periods (baseline to wk 8, wk 8-16, wk 16-24, and wk 24-32), with 19, 17, 16, and 15 unique surveys deployed, respectively. We used mixed-effects regression models with random intercepts to compare the repeated outcome between groups. Stratified analyses by age and sex were performed.

Among 492 participants (mean age 74, SD 6.3 y; 58%, n=284 women, 84%, n=413 non-Hispanic White), there was evidence that the experimental group had higher response rates over time compared to the control group (P=.003 for interaction between deployment pattern and time). Both groups showed similar proportions of surveys returned during the first period (75% vs 76%). The experimental group had higher response rates than the control group in subsequent periods (70% vs 67% in wk 8-16, 64% vs 59% in wk 16-24, and 58% vs 50% in wk 24-32). The proportion of participants not returning any surveys increased from 3% to 38% in the control group compared to 1% to 28% in the experimental group across the four time periods. Stratified analyses revealed that among younger participants (≤75 y), the experimental group showed 12% higher survey response rates compared to the control group in the final period, while the difference was minimal among older participants (>75 years). The effect of the deployment pattern was similar for men and women. Three-way interaction analyses revealed no significant differences in the deployment pattern effect over time by age group (P=.95) or sex (P=.65).

Administering half of the surveys every 2 weeks, as compared to all surveys every 4 weeks, was associated with higher maintained longitudinal survey response rates. This strategy may help mitigate survey fatigue and improve data quality in digital health studies.

ClinicalTrials.gov NCT04752657; https://clinicaltrials.gov/study/NCT04752657

Figures