Application of data engineering approaches to address challenges in microbiome data for optimal medical decision-making

TL;DR

This study applies data engineering techniques like SMOTE and PCA to microbiome data, improving classification accuracy and efficiency, thereby aiding personalized medicine through better handling of high-dimensional, imbalanced microbiome datasets.

Contribution

It introduces a data engineering pipeline combining SMOTE and PCA to enhance machine learning classification of microbiome data, addressing class imbalance and high dimensionality.

Findings

Ensemble classifiers (RF and XGB) outperform others in accuracy.

PCA reduces testing time significantly.

Highest accuracy achieved at species level.

Abstract

The human gut microbiota is known to contribute to numerous physiological functions of the body and also implicated in a myriad of pathological conditions. Prolific research work in the past few decades have yielded valuable information regarding the relative taxonomic distribution of gut microbiota. Unfortunately, the microbiome data suffers from class imbalance and high dimensionality issues that must be addressed. In this study, we have implemented data engineering algorithms to address the above-mentioned issues inherent to microbiome data. Four standard machine learning classifiers (logistic regression (LR), support vector machines (SVM), random forests (RF), and extreme gradient boosting (XGB) decision trees) were implemented on a previously published dataset. The issue of class imbalance and high dimensionality of the data was addressed through synthetic minority oversampling technique (SMOTE) and principal component analysis (PCA). Our results indicate that ensemble classifiers (RF and XGB decision trees) exhibit superior classification accuracy in predicting the host phenotype. The application of PCA significantly reduced testing time while maintaining high classification accuracy. The highest classification accuracy was obtained at the levels of species for most classifiers. The prototype employed in the study addresses the issues inherent to microbiome datasets and could be highly beneficial for providing personalized medicine.