# Implementing Digital Mental Health Tools in the Public Behavioral Health Systems: A Multi-City/County Evaluation of California’s Help@Hand Project

## TL;DR

This study evaluates how digital mental health tools were implemented in public systems across California, highlighting factors that helped or hindered success.

## Contribution

The study provides new insights into the implementation of digital mental health tools in public systems using a multi-site evaluation.

## Key findings

- Vendor support helped, but customization and data access were barriers.
- Workforce sustainability was challenging due to turnover and understaffing.
- Digital equity requires addressing device access, internet connectivity, and digital literacy.

## Abstract

Digital mental health interventions (DMHIs) can expand access to mental health care, particularly in underserved regions. Despite this, real-world implementation is complex and few studies have investigated the factors that shape it. We provide insights from Help@Hand, a five-year implementation project deploying DMHIs within public systems across counties and cities (i.e., sites) in California. This study examined how diverse local systems implemented DMHIs within a multi-site initiative and identified factors that facilitated or prevented success.

Data were collected from project leads at participating sites (N = 12) through four waves of surveys and two rounds of interviews. Surveys and interviews were informed by the Consolidated Framework for Implementation Research (CFIR) and captured implementation experiences over time, providing contextual insight into local decision-making, resource use, and coordination efforts. Interviews and surveys were analyzed using a sequential mixed-methods design using the CFIR. Reflexive thematic analysis and descriptive survey statistics were used to examine implementation strategies, challenges, and adaptations across sites.

Eight themes were identified: (1) vendors supplementation was a facilitator, though customization and data access often posed as barriers; (2) sustaining a workforce proved challenging due to turnover and understaffing; (3) sites operated in tight and uncertain funding environments; (4) community outreach required multimodal strategies, with in-person engagement more effective than traditional marketing; (5) new media proved essential for marketing, but was resource-intensive; (6) the project management entity provided contracting support, but introduced bureaucratic delays; (7) collaborations were both facilitated and disrupted by organizational factors; and (8) achieving digital equity is challenging, requiring considerations device access, internet connectivity, and digital literacy.

This study provides insights into the processes underpinning the implementation of DMHIs within public behavioral health systems. Successful implementation did not merely involve providing the community with access to digital tools; it required extensive coordination within and across sites, while ensuring that technologies could be adapted to meet local needs. Future large-scale digital mental health initiatives should aim to operationalize insights from this evaluation to ensure successful and sustainable DMHIs, with structured opportunities for collaboration and shared learning to enhance organizational capacity and alignment across levels.