Understanding Representation Gaps Across Scales in Tropical Tree Species Classification from Drone Imagery

TL;DR

This paper investigates the challenges of classifying tropical tree species from UAV imagery at different resolutions, highlighting the potential of self-supervised methods to bridge the performance gap.

Contribution

It evaluates existing models on paired UAV imagery at different scales and proposes self-supervised representation alignment to improve canopy-level classification.

Findings

Classification accuracy is higher on close-up images than on top-view imagery.

The performance gap widens for rare species.

Self-supervised alignment can integrate fine-grained details into canopy-level models.

Abstract

Accurate classification of tropical tree species from unoccupied aerial vehicle (UAV) imagery remains challenging due to high species diversity and strong visual similarity among species at typical image resolutions (centimeters per pixel). In contrast, models trained on close-up citizen science photographs captured with smartphones achieve strong plant species classification performance. Recent advances in UAV data acquisition now enable the collection of close-up images that are spatially registered with top-view aerial imagery and approach the level of visual detail found in smartphone photographs, with the trade-off that such high-resolution photos cannot be acquired for many trees. In this work, we evaluate the performance of existing methods using paired top-view and close-up UAV imagery collected in a species-rich tropical forest. Through fine-tuning experiments, we quantify the performance gap between vision foundation models and in-domain generalist plant recognition models across both image types (high-resolution close-up versus coarser-resolution top-view imagery). We show that classification performance is consistently higher on close-up images than on top-view aerial imagery, and that this performance gap widens for rare species. Finally, we propose that self-supervised representation alignment across these two spatial scales offers a promising approach for integrating fine-grained visual information into canopy-level species classification models based on top-view UAV imagery. Leveraging high-resolution close-up UAV imagery to enhance canopy-level species classification could substantially improve large-scale monitoring of tropical forest biodiversity.