Tree semantic segmentation from aerial image time series

TL;DR

This paper presents a novel approach for tree species semantic segmentation from aerial image time series, leveraging temporal data, a custom hierarchical loss, and pretrained models to improve accuracy and taxonomic consistency.

Contribution

It introduces a simple convolutional block for spatio-temporal feature extraction and a hierarchical loss function, enhancing segmentation performance using time series data and taxonomic information.

Findings

Time series models outperform single image models in segmentation accuracy.

Incorporating taxonomic hierarchy improves prediction consistency.

The proposed method effectively exploits temporal data for better tree species identification.

Abstract

Earth's forests play an important role in the fight against climate change, and are in turn negatively affected by it. Effective monitoring of different tree species is essential to understanding and improving the health and biodiversity of forests. In this work, we address the challenge of tree species identification by performing semantic segmentation of trees using an aerial image dataset spanning over a year. We compare models trained on single images versus those trained on time series to assess the impact of tree phenology on segmentation performances. We also introduce a simple convolutional block for extracting spatio-temporal features from image time series, enabling the use of popular pretrained backbones and methods. We leverage the hierarchical structure of tree species taxonomy by incorporating a custom loss function that refines predictions at three levels: species, genus, and higher-level taxa. Our findings demonstrate the superiority of our methodology in exploiting the time series modality and confirm that enriching labels using taxonomic information improves the semantic segmentation performance.