Does Your Wildfire Prediction Model Actually Work, or Just Score Well?

TL;DR

This paper introduces WILDFIRE-FM, a specialized foundation model for wildfire prediction, and proposes a fixed-contract evaluation framework to fairly compare models amidst sparse wildfire data.

Contribution

The paper presents WILDFIRE-FM, the first wildfire-specific foundation model, and a fixed-contract evaluation framework to improve benchmarking reliability.

Findings

Wildfire transfer conclusions are highly sensitive to evaluation design.

WILDFIRE-FM outperforms ten Earth-FM baselines in various tasks.

Evaluation settings significantly influence model performance assessments.

Abstract

Wildfire prediction is important for early warning and resource allocation, yet existing Earth foundation models (Earth FMs) are pretrained for general atmospheric and geophysical objectives rather than wildfire forecasting. To address this gap, we introduce WILDFIRE-FM, the first foundation model pretrained specifically for wildfire prediction using weather, active-fire observations, topography, vegetation, and static environmental data. However, introducing a domain-specific backbone alone does not solve the evaluation problem: wildfire events are sparse in space and time, making transfer conclusions highly sensitive to matching rules and evaluation settings. To address this problem, we introduce a fixed-contract evaluation framework with two controlled checks: a fixed-output check for matching-rule effects and a fixed-feature check for head-selection effects. Under matched contracts, we compare WILDFIRE-FM with ten Earth-FM baselines across occupancy, spread, retrieval, and regression tasks. Our results show that wildfire transfer conclusions depend strongly on evaluation design and task formulation. We hope this framework and WILDFIRE-FM provide a foundation for future wildfire-specific Earth-FM research and benchmarking. Our code is available at https://anonymous.4open.science/r/Wildfire-fm-evaluation-contracts-5AE9/.