Generalization and Feature Attribution in Machine Learning Models for Crop Yield and Anomaly Prediction in Germany

TL;DR

This paper investigates the generalization and interpretability of ML models for crop yield prediction in Germany, revealing that high test accuracy does not guarantee reliable explanations or temporal robustness.

Contribution

It systematically compares ensemble and deep learning models, highlighting limitations in generalization and the pitfalls of relying solely on post hoc interpretability methods.

Findings

Models perform well on spatial test sets but poorly on temporal validation.

High test accuracy models can still produce misleading feature importance.

Interpretability methods may not reflect true model generalization.

Abstract

This study examines the generalization performance and interpretability of machine learning (ML) models used for predicting crop yield and yield anomalies in Germany's NUTS-3 regions. Using a high-quality, long-term dataset, the study systematically compares the evaluation and temporal validation behavior of ensemble tree-based models (XGBoost, Random Forest) and deep learning approaches (LSTM, TCN). While all models perform well on spatially split, conventional test sets, their performance degrades substantially on temporally independent validation years, revealing persistent limitations in generalization. Notably, models with strong test-set accuracy, but weak temporal validation performance can still produce seemingly credible SHAP feature importance values. This exposes a critical vulnerability in post hoc explainability methods: interpretability may appear reliable even when the underlying model fails to generalize. These findings underscore the need for validation-aware interpretation of ML predictions in agricultural and environmental systems. Feature importance should not be accepted at face value unless models are explicitly shown to generalize to unseen temporal and spatial conditions. The study advocates for domain-aware validation, hybrid modeling strategies, and more rigorous scrutiny of explainability methods in data-driven agriculture. Ultimately, this work addresses a growing challenge in environmental data science: how can we evaluate generalization robustly enough to trust model explanations?