# PICO-based assessment and categorization of evidence for digital health interventions: an inductive framework development

## TL;DR

This paper introduces a framework to efficiently assess and categorize evidence from digital health interventions using abstract-level data for better evidence mapping.

## Contribution

The novel PACE4DHI framework enables structured screening of digital health intervention reviews at the abstract level for evidence and gap mapping.

## Key findings

- The PACE4DHI framework includes 41 categories covering problems, interventions, settings, outcomes, and evidence levels.
- Abstracts provided more detailed information on problems and interventions than on comparators and outcomes.
- The framework's accuracy was higher for conclusive evidence classifications than for inconclusive ones.

## Abstract

Despite the increasing number of systematic reviews on digital health interventions (DHIs), clear and robust evidence remains elusive due to methodological shortcomings in formulating research questions and conducting search and screening processes. The growing volume of reviews necessitates higher-level syntheses like umbrella reviews and evidence gap maps, requiring methods for rapid, systematic evidence assessment at the abstract level.

With the development of the PICO-based Assessment and Categorization of Evidence for Digital Health Interventions (PACE4DHI) framework we aim to enable the efficient structured screening of systematic reviews and meta-analyses at the level of abstracts for subsequent evidence and gap mapping (EGM).

A comprehensive literature search was performed across five databases, adhering to PRISMA guidelines, to capture systematic reviews and meta-analyses published between 2011 and October 2023. All categories of DHIs, populations, settings, and outcomes were considered. From 21,161 results, we screened 9,030 titles and abstracts post-de-duplication, with 2,528 remaining. To construct the framework, thematic analysis was conducted on a random sample of 250 studies. The framework's accuracy was validated on 138 open-access articles through full-text comparisons.

The PACE4DHI framework encompasses 41 categories, spanning 11 problems (e.g., cardiovascular diseases), 13 DHIs (e.g., telemedicine), 6 comparative care settings (e.g., outpatient care), 7 outcome dimensions (e.g., effectiveness), and 4 evidence classification levels. The PICO-categorization and evidence classification was confirmed with varying accuracy and largely consistent results at both abstract and full-text levels. Variability in the accuracy reflects that abstracts provided more detail on problems and interventions than they did for the comparator and outcomes. The likelihood of conclusive evidence was more accurately predicted for cardinal classes (high and low) than for inconclusiveness.

The PACE4DHI framework provides a systematic and pragmatic methodology, with potential to enhance structured access to existing evidence. The framework may also inform the research questions and the search and screening strategies of future systematic reviews. The application in EGM has potential to optimize evidence-based decision-making, while also enabling precise identification of research gaps. Its use with artificial intelligence tools may facilitate efficient ongoing evidence screening and synthesis, ultimately supporting a searchable evidence database.