DeepForest: Sensing Into Self-Occluding Volumes of Vegetation With Aerial Imaging

TL;DR

This paper introduces a novel aerial imaging method that uses synthetic-aperture imaging and neural networks to sense deep into dense vegetation canopies, providing detailed 3D vegetation data beyond traditional optical limits.

Contribution

It presents a new approach combining drone-based focal stack scanning and deep learning to penetrate dense vegetation, surpassing existing remote sensing techniques like LiDAR and radar.

Findings

Achieved approximately 7x improvement in forest density measurement accuracy.

Attained an MSE of 0.05 compared to classical multispectral imaging.

Successfully sensed deep into self-occluding vegetation volumes.

Abstract

Access to below-canopy volumetric vegetation data is crucial for understanding ecosystem dynamics. We address the long-standing limitation of remote sensing to penetrate deep into dense canopy layers. LiDAR and radar are currently considered the primary options for measuring 3D vegetation structures, while cameras can only extract the reflectance and depth of top layers. Using conventional, high-resolution aerial images, our approach allows sensing deep into self-occluding vegetation volumes, such as forests. It is similar in spirit to the imaging process of wide-field microscopy, but can handle much larger scales and strong occlusion. We scan focal stacks by synthetic-aperture imaging with drones and reduce out-of-focus signal contributions using pre-trained 3D convolutional neural networks with mean squared error (MSE) as the loss function. The resulting volumetric reflectance stacks contain low-frequency representations of the vegetation volume. Combining multiple reflectance stacks from various spectral channels provides insights into plant health, growth, and environmental conditions throughout the entire vegetation volume. Compared with simulated ground truth, our correction leads to ~x7 average improvements (min: ~x2, max: ~x12) for forest densities of 220 trees/ha - 1680 trees/ha. In our field experiment, we achieved an MSE of 0.05 when comparing with the top-vegetation layer that was measured with classical multispectral aerial imaging.