Using multi-method approaches to document and assess adaptations in a community-driven COVID-19 testing program

TL;DR

This paper describes a method for tracking and evaluating changes made during a community-driven COVID-19 testing program to improve its effectiveness and equity.

Contribution

A systematic, multi-method approach to document and assess adaptations in a real-world public health intervention.

Findings

Eighty-four unique adaptations were documented, with most occurring during pre-implementation.

Most adaptations were perceived as positive, with unexpected ones often improving reach and effectiveness.

Adaptations were generally small in scope, affecting less than 10% of core elements.

Abstract

Adaptations are expected when complex public health interventions are implemented in dynamically and rapidly changing real-world settings. Systematic documentation of adaptations to intervention components and strategies are critical when assessing their impact on implementation. The purpose of this paper is to describe our approach to systematically tracking, documenting, and evaluating adaptations made during the CO-CREATE-Ex project, which aimed to address COVID-19 testing disparities in the San Ysidro US/Mexico border community.

The study utilized a longitudinal, prospective, multi- method approach to systematically document and assess adaptations across the pre-implementation, early and mid/late-implementation, and maintenance phases of the project. Adaptations were aggregated from a combination of sources (i.e., meeting notes, Advisory Board transcripts, and periodic reflections). Adaptations were entered weekly into an electronic database that captured information on 16 characteristics and were validated by study staff. Descriptive statistics were used to describe adaptation characteristics. Adaptation impact was evaluated using a combination of objective and subjective measures aligned with the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) outcomes.

Eighty-four unique adaptations were included in this analysis. Adaptations were organized by study phase with most occurring during pre-implementation. Most adaptations (n = 79, 94.04%) were planned (i.e., proactive) and expected (n = 63, 75%), and (n = 21, 25.0%) adaptations were considered unexpected (e.g., reactive). Across all adaptations, 71.2% were perceived as positive (i.e., had a positive impact on RE-AIM implementation outcomes) and 19.1% were perceived to be negative (i.e., worsened implementation outcome or decreased implementation). Unexpected adaptations, though reactive in nature, generally had a positive impact on implementation outcomes. For instance, 14.3% of unexpected adaptations were perceived to increase reach and effectiveness. Within maintenance, 19% of unexpected adaptations were perceived to increase this outcome. Lastly, adaptations were generally small in scope with less than a tenth of adaptations affecting 50% or more of core elements.

Our systematic approach to documenting and analyzing adaptations has highlighted the importance of understanding the impact of adaptations on implementation outcomes. These insights underscore the need for continued research to refine methods for adaptation documentation and impact evaluation, ensuring interventions remain effective, equitable, and responsive to real-world challenges.

ClinicalTrials.gov, NCT05894655, Registered 8 June 2023.

Figures