DNN-based 3D Cloud Retrieval for Variable Solar Illumination and Multiview Spaceborne Imaging

TL;DR

This paper presents a scalable deep neural network system for 3D cloud property retrieval from multiview satellite images, effectively handling variable solar illumination and camera poses to improve volumetric cloud analysis.

Contribution

The work introduces the first scalable DNN-based method for 3D cloud retrieval that accommodates changing solar directions and camera poses, with a novel two-stage training scheme.

Findings

Significant improvement over previous methods in handling solar zenith angle variations.

Effective integration of multiview images, camera poses, and solar data.

Enhanced scalability for large-scale climate data analysis.

Abstract

Climate studies often rely on remotely sensed images to retrieve two-dimensional maps of cloud properties. To advance volumetric analysis, we focus on recovering the three-dimensional (3D) heterogeneous extinction coefficient field of shallow clouds using multiview remote sensing data. Climate research requires large-scale worldwide statistics. To enable scalable data processing, previous deep neural networks (DNNs) can infer at spaceborne remote sensing downlink rates. However, prior methods are limited to a fixed solar illumination direction. In this work, we introduce the first scalable DNN-based system for 3D cloud retrieval that accommodates varying camera poses and solar directions. By integrating multiview cloud intensity images with camera poses and solar direction data, we achieve greater flexibility in recovery. Training of the DNN is performed by a novel two-stage scheme to address the high number of degrees of freedom in this problem. Our approach shows substantial improvements over previous state-of-the-art, particularly in handling variations in the sun's zenith angle.