Improving the quality of Persian clinical text with a novel spelling correction system

TL;DR

This paper presents a novel spelling correction system for Persian clinical text that combines a fine-tuned pre-trained model with an innovative orthographic similarity algorithm, achieving high accuracy in error detection and correction.

Contribution

The study introduces a new approach integrating a pre-trained model and the PERTO algorithm for improved Persian clinical text spelling correction, addressing language-specific challenges.

Findings

F1-Score of 90.0% for non-word error correction with PERTO

F1-Score of 90.6% for real-word error detection

F1-Score of 91.5% for real-word correction with PERTO

Abstract

Background: The accuracy of spelling in Electronic Health Records (EHRs) is a critical factor for efficient clinical care, research, and ensuring patient safety. The Persian language, with its abundant vocabulary and complex characteristics, poses unique challenges for real-word error correction. This research aimed to develop an innovative approach for detecting and correcting spelling errors in Persian clinical text. Methods: Our strategy employs a state-of-the-art pre-trained model that has been meticulously fine-tuned specifically for the task of spelling correction in the Persian clinical domain. This model is complemented by an innovative orthographic similarity matching algorithm, PERTO, which uses visual similarity of characters for ranking correction candidates. Results: The evaluation of our approach demonstrated its robustness and precision in detecting and rectifying word errors in Persian clinical text. In terms of non-word error correction, our model achieved an F1-Score of 90.0% when the PERTO algorithm was employed. For real-word error detection, our model demonstrated its highest performance, achieving an F1-Score of 90.6%. Furthermore, the model reached its highest F1-Score of 91.5% for real-word error correction when the PERTO algorithm was employed. Conclusions: Despite certain limitations, our method represents a substantial advancement in the field of spelling error detection and correction for Persian clinical text. By effectively addressing the unique challenges posed by the Persian language, our approach paves the way for more accurate and efficient clinical documentation, contributing to improved patient care and safety. Future research could explore its use in other areas of the Persian medical domain, enhancing its impact and utility.