Learning Image-based Tree Crown Segmentation from Enhanced Lidar-based Pseudo-labels

TL;DR

This paper introduces a deep learning approach for segmenting individual tree crowns in aerial imagery by leveraging enhanced pseudo-labels from ALS data and the SAM 2 model, eliminating manual annotation costs.

Contribution

The study presents a novel method to generate domain-specific training data for tree crown segmentation using pseudo-labels and zero-shot models, improving performance over existing general models.

Findings

Pseudo-labels from ALS can be effectively used for training.

Enhanced pseudo-labels improve segmentation accuracy.

Method outperforms existing general domain models.

Abstract

Mapping individual tree crowns is essential for tasks such as maintaining urban tree inventories and monitoring forest health, which help us understand and care for our environment. However, automatically separating the crowns from each other in aerial imagery is challenging due to factors such as the texture and partial tree crown overlaps. In this study, we present a method to train deep learning models that segment and separate individual trees from RGB and multispectral images, using pseudo-labels derived from aerial laser scanning (ALS) data. Our study shows that the ALS-derived pseudo-labels can be enhanced using a zero-shot instance segmentation model, Segment Anything Model 2 (SAM 2). Our method offers a way to obtain domain-specific training annotations for optical image-based models without any manual annotation cost, leading to segmentation models which outperform any available models which have been targeted for general domain deployment on the same task.