Data-Driven Regular Expressions Evolution for Medical Text Classification Using Genetic Programming

TL;DR

This paper introduces a genetic programming approach to evolve interpretable regular expressions for medical text classification, enhancing transparency and allowing medical professionals to fine-tune classifiers for better accuracy.

Contribution

It presents a novel method combining genetic programming with a new regular expression syntax to produce understandable classifiers for medical text, improving interpretability and customization.

Findings

Effective classification of medical inquiries with satisfactory precision and recall

Generated classifiers are fully understandable and editable by medical professionals

Promising performance demonstrated on real healthcare data

Abstract

In medical fields, text classification is one of the most important tasks that can significantly reduce human workload through structured information digitization and intelligent decision support. Despite the popularity of learning-based text classification techniques, it is hard for human to understand or manually fine-tune the classification results for better precision and recall, due to the black box nature of learning. This study proposes a novel regular expression-based text classification method making use of genetic programming (GP) approaches to evolve regular expressions that can classify a given medical text inquiry with satisfactory precision and recall while allow human to read the classifier and fine-tune accordingly if necessary. Given a seed population of regular expressions (can be randomly initialized or manually constructed by experts), our method evolves a population of regular expressions according to chosen fitness function, using a novel regular expression syntax and a series of carefully chosen reproduction operators. Our method is evaluated with real-life medical text inquiries from an online healthcare provider and shows promising performance. More importantly, our method generates classifiers that can be fully understood, checked and updated by medical doctors, which are fundamentally crucial for medical related practices.