Human Limits in Machine Learning: Prediction of Plant Phenotypes Using Soil Microbiome Data

TL;DR

This study investigates the potential of machine learning models, specifically random forest and Bayesian neural networks, to predict plant phenotypes from soil microbiome and environmental data, highlighting the importance of data preprocessing and human decision-making.

Contribution

It is the first comprehensive analysis of soil microbiome data for plant phenotype prediction, emphasizing the impact of data normalization and human choices on model performance.

Findings

Incorporating environmental features improves prediction accuracy.

Naive normalization methods like total sum scaling are suboptimal.

Accurate label definition is more critical than normalization or model choice.

Abstract

The preservation of soil health is a critical challenge in the 21st century due to its significant impact on agriculture, human health, and biodiversity. We provide the first deep investigation of the predictive potential of machine learning models to understand the connections between soil and biological phenotypes. We investigate an integrative framework performing accurate machine learning-based prediction of plant phenotypes from biological, chemical, and physical properties of the soil via two models: random forest and Bayesian neural network. We show that prediction is improved when incorporating environmental features like soil physicochemical properties and microbial population density into the models, in addition to the microbiome information. Exploring various data preprocessing strategies confirms the significant impact of human decisions on predictive performance. We show that the naive total sum scaling normalization that is commonly used in microbiome research is not the optimal strategy to maximize predictive power. Also, we find that accurately defined labels are more important than normalization, taxonomic level or model characteristics. In cases where humans are unable to classify samples accurately, machine learning model performance is limited. Lastly, we provide domain scientists via a full model selection decision tree to identify the human choices that optimize model prediction power. Our work is accompanied by open source reproducible scripts (https://github.com/solislemuslab/soil-microbiome-nn) for maximum outreach among the microbiome research community.