Multimodal Object Detection Under Sparse Forest-Canopy Occlusion

TL;DR

This paper explores a multimodal detection pipeline combining LiDAR, visible-thermal fusion, and synthetic imaging to improve human detection under forest canopy, providing initial benchmarks for UAV-based rescue and surveillance.

Contribution

It introduces a novel multimodal pipeline integrating LiDAR, image fusion, and synthetic imaging for detection in forest environments, with a fine-tuned YOLOv5 detector achieving high accuracy.

Findings

LiDAR penetration is limited for object detection in forests.

Fusion of visible and thermal images enhances target visibility.

Synthetic aperture imaging improves ground-plane detection.

Abstract

Reliable detection of humans beneath forest canopy remains a difficult remote-sensing challenge due to sparse, structured, and viewpoint-dependent occlusion. This paper presents a multimodal proof-of-concept pipeline that integrates three complementary approaches: (i) experimental evaluation of LiDAR returns through vegetation to assess the feasibility of active sensing, (ii) visible--thermal image fusion using a multi-scale transform and sparse-representation framework to enhance human saliency, and (iii) synthetic-aperture image formation via Airborne Optical Sectioning (AOS) to suppress canopy clutter. A YOLOv5 detector is fine-tuned on the Teledyne FLIR thermal dataset and evaluated on thermal and fused imagery. Results show that the tested terrestrial LiDAR configuration provides limited penetration for object-level detection, while visible--thermal fusion improves target visibility in low-contrast scenes and AOS enhances ground-plane detection in synthetic forest imagery. The fine-tuned YOLOv5 achieves a mean average precision of $\sim$0.83 on the top three FLIR classes. These findings establish an initial baseline for UAV-deployable search-and-rescue and surveillance systems operating in forested environments, and motivate future work on dedicated forest datasets and real-time multimodal integration.