Hand-drawn Symbol Recognition of Surgical Flowsheet Graphs with Deep Image Segmentation

TL;DR

This paper introduces a deep learning-based image segmentation method using U-Net to accurately recognize hand-drawn symbols on surgical flowsheets, enabling digitization of perioperative data in resource-limited settings.

Contribution

The study presents a novel application of U-Net for symbol detection on surgical flowsheets, outperforming template matching with limited training data.

Findings

Achieved over 99% accuracy in symbol detection.

More than 95% of predictions within an absolute error of five.

Outperformed traditional template matching methods.

Abstract

Perioperative data are essential to investigating the causes of adverse surgical outcomes. In some low to middle income countries, these data are computationally inaccessible due to a lack of digitization of surgical flowsheets. In this paper, we present a deep image segmentation approach using a U-Net architecture that can detect hand-drawn symbols on a flowsheet graph. The segmentation mask outputs are post-processed with techniques unique to each symbol to convert into numeric values. The U-Net method can detect, at the appropriate time intervals, the symbols for heart rate and blood pressure with over 99 percent accuracy. Over 95 percent of the predictions fall within an absolute error of five when compared to the actual value. The deep learning model outperformed template matching even with a small size of annotated images available for the training set.