Adaptation of Autoencoder for Sparsity Reduction From Clinical Notes Representation Learning

TL;DR

This paper introduces an autoencoder-based method to reduce sparsity in clinical note representations, improving classification performance on small datasets with high-dimensional, sparse features.

Contribution

It proposes a novel autoencoder approach for compressing sparse clinical text features, enhancing classification accuracy and demonstrating the method within the information bottleneck framework.

Findings

Performance gains of up to 3% in classification metrics.

Achieved 92% accuracy and 91% recall in patient condition detection.

Validated the compression mechanism using the information bottleneck theory.

Abstract

When dealing with clinical text classification on a small dataset recent studies have confirmed that a well-tuned multilayer perceptron outperforms other generative classifiers, including deep learning ones. To increase the performance of the neural network classifier, feature selection for the learning representation can effectively be used. However, most feature selection methods only estimate the degree of linear dependency between variables and select the best features based on univariate statistical tests. Furthermore, the sparsity of the feature space involved in the learning representation is ignored. Goal: Our aim is therefore to access an alternative approach to tackle the sparsity by compressing the clinical representation feature space, where limited French clinical notes can also be dealt with effectively. Methods: This study proposed an autoencoder learning algorithm to take advantage of sparsity reduction in clinical note representation. The motivation was to determine how to compress sparse, high-dimensional data by reducing the dimension of the clinical note representation feature space. The classification performance of the classifiers was then evaluated in the trained and compressed feature space. Results: The proposed approach provided overall performance gains of up to 3% for each evaluation. Finally, the classifier achieved a 92% accuracy, 91% recall, 91% precision, and 91% f1-score in detecting the patient's condition. Furthermore, the compression working mechanism and the autoencoder prediction process were demonstrated by applying the theoretic information bottleneck framework.