Development of a hybrid automated chart checking, data collection, and analysis system

TL;DR

This paper describes a hybrid system for automated chart checking in oncology that improves efficiency and accuracy in plan reviews.

Contribution

The novel concept of 'plan attributes' allows for dynamic checklist generation tailored to specific treatment plans.

Findings

The system processed over 427,073 tests across 7,882 chart checks over four years.

It identified 7,116 minor errors, 679 major errors, and 71 plan rejections.

The hybrid approach combines automation with human input to enhance chart check performance.

Abstract

To describe the creation and clinical deployment of an integrated stand‐alone hybrid automated chart review platform that is comprised of a database, oncology information system (OIS) interfaces, and a user interface (UI).

A consensus checklist was developed to homogenize the practices of clinical physics staff and physics residents for all plan types. Critical review of the single checklist's items led to the creation of a new concept, the “plan attribute,” which permits specialization of the final checklist based on the characteristics of a particular plan. Only relevant and applicable tests are automatically populated to form a plan checklist based on plan attributes. Each populated checklist in this hybrid automation platform includes elements of full automation, steps or functions that use automation to augment a human input or effort, as well as steps completed by humans. Logging and categorizing of deficiencies found during the chart check was also implemented.

Chart check data was analyzed for an over 4‐year time span starting from the clinical deployment of the tool in October 2020. This analysis included a total of 427 073 tests spread among 7882 initial chart checks. The recorded deficiencies at three triage levels showed 7116 minor errors, 679 major errors, and 71 plan rejections. The plan rejection level is the highest chart deficiency, which required the plan to be rejected and sent back for serious remediation and/or new plan generation.

This report describes the successful development and implementation of a hybrid‐automated chart checking program. The software leverages modern database and software design to improve the per case performance and experience of a multi‐user group. Chart check database permits high‐level review and improvement in all aspects of plan check and user performance.

Figures