Meaningfully meeting the interoperability mandate: a review of the Assistant Secretary for Technology Policy Real World Testing practices

TL;DR

This study finds that health IT developers are not providing clear, contextualized interoperability results as required by U.S. regulations.

Contribution

The paper reveals a gap in how developers report interoperability results under the ONC certification program.

Findings

Developers provided interoperability measures but lacked meaningful context in their reporting.

Certified health IT modules do not meet transparency requirements for interoperability as intended by the 21st Century Cures Act.

Clearer guidance and stricter review of Real World Testing reports may be necessary.

Abstract

We analyzed interoperability-related Real World Testing results to identify whether developers are providing meaningful results with the appropriate context to enable stakeholders to understand the Certified Health IT conformance and interoperability when deployed in production environments.

This qualitative study analyzed components of the Assistant Secretary for Technology Policy’s transitions of care criterion Real World Testing results of 5 inpatient and 5 ambulatory health IT developers with the largest market share.

Developers provided interoperability measures; however, none of the developers’ presented results in a meaningful way with the appropriate context to understand product interoperability.

Our results suggest that developers with ASTP/Office of the National Coordinator (ONC) Certified Health IT modules are not providing interoperability transparency through Real World Testing as required by the ONC Health IT Certification Program and intended by the 21st Century Cures Act.

Clearer developer guidance and actual metric requirements on Real World Testing may be required and the authorized certification bodies, who review developer results, may need to more closely inspect reports to look at the quality of reported results.