Reliability-based cleaning of noisy training labels with inductive conformal prediction in multi-modal biomedical data mining

TL;DR

This paper introduces a reliability-based data cleaning method using inductive conformal prediction to improve classification accuracy in noisy, multi-modal biomedical datasets without requiring extensive manual labeling.

Contribution

The proposed method leverages ICP-derived reliability metrics to identify and correct mislabeled data, enhancing classification performance across diverse biomedical modalities.

Findings

Significant accuracy improvements in drug-induced liver injury classification

Enhanced AUROC and AUPRC in COVID-19 patient prediction

High accuracy and F1 score gains in breast cancer subtyping

Abstract

Accurately labeling biomedical data presents a challenge. Traditional semi-supervised learning methods often under-utilize available unlabeled data. To address this, we propose a novel reliability-based training data cleaning method employing inductive conformal prediction (ICP). This method capitalizes on a small set of accurately labeled training data and leverages ICP-calculated reliability metrics to rectify mislabeled data and outliers within vast quantities of noisy training data. The efficacy of the method is validated across three classification tasks within distinct modalities: filtering drug-induced-liver-injury (DILI) literature with title and abstract, predicting ICU admission of COVID-19 patients through CT radiomics and electronic health records, and subtyping breast cancer using RNA-sequencing data. Varying levels of noise to the training labels were introduced through label permutation. Results show significant enhancements in classification performance: accuracy enhancement in 86 out of 96 DILI experiments (up to 11.4%), AUROC and AUPRC enhancements in all 48 COVID-19 experiments (up to 23.8% and 69.8%), and accuracy and macro-average F1 score improvements in 47 out of 48 RNA-sequencing experiments (up to 74.6% and 89.0%). Our method offers the potential to substantially boost classification performance in multi-modal biomedical machine learning tasks. Importantly, it accomplishes this without necessitating an excessive volume of meticulously curated training data.