# Transformer model to determine spatio-temporal relationships of variables, and interpretability for soybean seed yield, oil, and protein prediction

## TL;DR

A Transformer model predicts soybean seed yield, oil, and protein using 30 years of data, showing high accuracy and identifying key environmental factors.

## Contribution

A novel Transformer-based framework for soybean trait prediction using multi-environment data and interpretability analysis.

## Key findings

- The model achieved R2 scores of 77.6%, 63.9%, and 79.3% for seed yield, oil, and protein prediction.
- Solar radiation and temperature were identified as key predictors for seed yield.
- The model captures complex temporal patterns in trait variability across diverse environments.

## Abstract

Accurate in-season prediction of seed yield and seed composition traits such as oil and protein are useful for gaining accuracy and efficiency in soybean breeding. These predictions can also inform farmers, enabling them to improve their field management practices, and guide their market decisions. We report a Transformer-based deep learning framework built on 30 years of multi-environment performance data from the Northern and Southern Uniform Soybean Tests (UST) across North America. Unlike earlier studies on seed yield, oil and protein prediction that focus on limited years, regions, single modalities, we utilized a comprehensive dataset that includes weather, genotype, and management factors, ensuring a more holistic approach to soybean yield, oil, and protein prediction. Our model integrates multivariate time-series weather data with genotypic relationship information, maturity group, and geographic location, to predict variety performance in diverse environments. Our model captures complex temporal patterns associated with trait variability; showing high predictive accuracy (R2) of 77.6 ± 0.2%, 63.9 ± 4.7%, and 79.3 ± 2.3% for seed yield, oil, and protein, respectively. Additionally, for seed yield, we also evaluated multiple interpretability methods to assess feature importance for predictor variables and critical growing timepoints, and solar radiation and temperature were noted as the key predictors. Overall, these results demonstrate the usefulness of a Transformer-based model in trait predictions, and the utility of large cooperative datasets from breeding programs.