Co-Created Digital Pretherapy Psychoeducation for Outpatients in Specialized Mental Health Care: Usability Evaluation and Patient Satisfaction Study

TL;DR

Researchers developed and tested StartHelp, a digital pretherapy program for mental health patients, finding it usable but needing navigation improvements.

Contribution

The paper introduces StartHelp, a co-created digital pretherapy tool, and evaluates its usability for future clinical trials.

Findings

StartHelp's video content showed good usability with a mean SUS score of 83.7.

Navigation of the app's infrastructure had lower usability scores (mean 63.6).

Patients rated the videos as high quality and nonoffensive despite moderate satisfaction scores.

Abstract

Specialized mental health services are facing high demand, potentially leading to lower-quality care. One solution may be to prepare patients for attending treatment. Digital pretherapy psychoeducation may be particularly relevant. However, the effectiveness of such an intervention depends on user engagement and satisfaction, and usability is therefore one of the most important factors.

This article has 2 objectives. Study 1 describes the development of StartHelp, a co-created digital pretherapy psychoeducation program for patients on waiting lists before their first consultation in outpatient specialized mental health services. Study 2 explores the usability of StartHelp, aiming to identify potential issues and assess whether the intervention is suitable for further evaluation in a randomized controlled trial.

Guided by co-creation principles, we developed StartHelp in accordance with the Guidance for Reporting Involvement of Patients and the Public (GRIPP2) checklist. To assess the app’s usability, we recruited 10 patients from specialized mental health care to complete tasks during individual think-aloud interviews. Afterward, they completed questionnaires, including open-ended questions, an item assessing perceived video quality, 2 versions of the System Usability Scale (SUS), the 4-item Client Satisfaction Questionnaire (CSQ-4), and a modified version of the CSQ-4 (CSQ-Video). The StartHelp project group discussed the results, and solutions to the identified issues were proposed and implemented.

Study 1 involved the development of StartHelp over 24 months. The app comprises 27 tasks, including 24 videos and links to 14 websites containing in-depth information. Study 2 involved usability testing with 5 men and 5 women. SUS scores for StartHelp’s videos indicated good usability, with a mean of 83.7. By contrast, SUS scores related to navigating StartHelp’s overarching architecture were barely acceptable, with a mean of 63.6. For the CSQ-4, the sample mean was 12.3, indicating moderate satisfaction. Mean scores on the CSQ-Video (10.9) indicated satisfaction in the lower moderate range. However, patients perceived the videos as high quality and rated them as nonoffensive. The qualitative findings supported the quantitative results. The usability tests revealed 1 major issue and several minor issues. The primary issue concerned navigation of the overarching technological infrastructure on which StartHelp was developed, rather than StartHelp itself. To address these issues and impracticalities related to interacting with the overarching technological infrastructure, we made minor changes to the StartHelp app.

Through a collaborative co-creation process, we developed StartHelp, a digital pretherapy psychoeducation program. Usability testing indicated that the content itself was highly usable, with video-related SUS scores suggesting good usability, whereas navigation of the overarching technological infrastructure received lower scores. Although patients rated the clinical material as high quality and nonoffensive, satisfaction scores were lower than anticipated. Nevertheless, StartHelp was deemed ready for further testing in a clinical trial.

Figures