From Articles to Canopies: Knowledge-Driven Pseudo-Labelling for Tree Species Classification using LLM Experts

TL;DR

This paper introduces a biologically informed semi-supervised deep learning approach that combines hyperspectral and LiDAR data with ecological knowledge via LLM-derived priors to improve tree species classification.

Contribution

It presents a novel integration of ecological priors from LLMs into pseudo-labelling for hyperspectral tree classification, enhancing accuracy with low training costs.

Findings

Achieved 5.6% higher accuracy than the best reference method.

Derived ecological cohabitation priors with high accuracy, within 15% of expert evaluations.

Demonstrated effectiveness on real-world forest datasets.

Abstract

Hyperspectral tree species classification is challenging due to limited and imbalanced class labels, spectral mixing (overlapping light signatures from multiple species), and ecological heterogeneity (variability among ecological systems). Addressing these challenges requires methods that integrate biological and structural characteristics of vegetation, such as canopy architecture and interspecific interactions, rather than relying solely on spectral signatures. This paper presents a biologically informed, semi-supervised deep learning method that integrates multi-sensor Earth observation data, specifically hyperspectral imaging (HSI) and airborne laser scanning (ALS), with expert, ecological knowledge. The approach relies on biologically inspired pseudo-labelling over a precomputed canopy graph, yielding accurate classification at low training cost. In addition, ecological priors on species cohabitation are automatically derived from reliable sources using large language models (LLMs) and encoded as a cohabitation matrix with likelihoods of species occurring together. These priors are incorporated into the pseudo-labelling strategy, effectively introducing expert knowledge into the model. Experiments on a real-world forest dataset demonstrate 5.6% improvement over the best reference method. Expert evaluation of cohabitation priors reveals high accuracy with differences no larger than 15%.