Learning Regular Expressions for Interpretable Medical Text Classification Using a Pool-based Simulated Annealing and Word-vector Models

TL;DR

This paper introduces an automated method for generating interpretable regular expressions for medical text classification, combining heuristic construction with Pool-based Simulated Annealing to reduce manual effort and improve interpretability.

Contribution

The paper presents a novel automated approach for creating high-quality, interpretable regular expressions for medical NLP tasks, reducing manual labor compared to traditional methods.

Findings

Automated regular expressions achieve high classification accuracy.

Method reduces manual effort in rule creation.

Regular expressions are interpretable and suitable for medical applications.

Abstract

In this paper, we propose a rule-based engine composed of high quality and interpretable regular expressions for medical text classification. The regular expressions are auto generated by a constructive heuristic method and optimized using a Pool-based Simulated Annealing (PSA) approach. Although existing Deep Neural Network (DNN) methods present high quality performance in most Natural Language Processing (NLP) applications, the solutions are regarded as uninterpretable black boxes to humans. Therefore, rule-based methods are often introduced when interpretable solutions are needed, especially in the medical field. However, the construction of regular expressions can be extremely labor-intensive for large data sets. This research aims to reduce the manual efforts while maintaining high-quality solutions