An AI Implementation Science Study to Improve Trustworthy Data in a Large Healthcare System

TL;DR

This paper demonstrates how integrating Trustworthy AI principles into data quality assessment and hybrid implementation strategies can enhance AI adoption in a large healthcare system, focusing on pediatric data management.

Contribution

It introduces a novel data quality assessment tool aligned with Trustworthy AI principles and compares systematic versus case-specific AI implementation strategies in healthcare.

Findings

Enhanced data quality evaluation addressing missingness and redundancy

Successful modernization of data infrastructure to OMOP CDM v5.4

Insights into hybrid AI implementation strategies in clinical settings

Abstract

The rapid growth of Artificial Intelligence (AI) in healthcare has sparked interest in Trustworthy AI and AI Implementation Science, both of which are essential for accelerating clinical adoption. However, strict regulations, gaps between research and clinical settings, and challenges in evaluating AI systems continue to hinder real-world implementation. This study presents an AI implementation case study within Shriners Childrens (SC), a large multisite pediatric system, showcasing the modernization of SCs Research Data Warehouse (RDW) to OMOP CDM v5.4 within a secure Microsoft Fabric environment. We introduce a Python-based data quality assessment tool compatible with SCs infrastructure, extending OHDsi's R/Java-based Data Quality Dashboard (DQD) and integrating Trustworthy AI principles using the METRIC framework. This extension enhances data quality evaluation by addressing informative missingness, redundancy, timeliness, and distributional consistency. We also compare systematic and case-specific AI implementation strategies for Craniofacial Microsomia (CFM) using the FHIR standard. Our contributions include a real-world evaluation of AI implementations, integration of Trustworthy AI principles into data quality assessment, and insights into hybrid implementation strategies that blend systematic infrastructure with use-case-driven approaches to advance AI in healthcare.