Cloud4D: Estimating Cloud Properties at a High Spatial and Temporal Resolution

TL;DR

Cloud4D is a novel machine learning framework that reconstructs high-resolution, four-dimensional cloud properties and wind vectors using synchronized ground-based cameras, significantly surpassing satellite measurement resolution.

Contribution

It introduces the first learning-based method for high-resolution 4D cloud state reconstruction from ground cameras, enabling detailed cloud and wind analysis.

Findings

Achieves 25 m spatial and 5 s temporal resolution in cloud reconstruction.

Provides less than 10% error compared to radar measurements.

Outperforms satellite data by an order of magnitude in resolution.

Abstract

There has been great progress in improving numerical weather prediction and climate models using machine learning. However, most global models act at a kilometer-scale, making it challenging to model individual clouds and factors such as extreme precipitation, wind gusts, turbulence, and surface irradiance. Therefore, there is a need to move towards higher-resolution models, which in turn require high-resolution real-world observations that current instruments struggle to obtain. We present Cloud4D, the first learning-based framework that reconstructs a physically consistent, four-dimensional cloud state using only synchronized ground-based cameras. Leveraging a homography-guided 2D-to-3D transformer, Cloud4D infers the full 3D distribution of liquid water content at 25 m spatial and 5 s temporal resolution. By tracking the 3D liquid water content retrievals over time, Cloud4D additionally estimates horizontal wind vectors. Across a two-month deployment comprising six skyward cameras, our system delivers an order-of-magnitude improvement in space-time resolution relative to state-of-the-art satellite measurements, while retaining single-digit relative error ($<10\%$) against collocated radar measurements. Code and data are available on our project page https://cloud4d.jacob-lin.com/.