OpenWildlife: Open-Vocabulary Multi-Species Wildlife Detector for Geographically-Diverse Aerial Imagery

TL;DR

OpenWildlife is an open-vocabulary wildlife detection system that uses language-aware embeddings and a novel framework adaptation to identify multiple species across diverse aerial imagery, outperforming existing methods.

Contribution

It introduces a flexible, language-aware wildlife detector leveraging a new adaptation of Grounding-DINO, capable of recognizing species via natural language in diverse environments.

Findings

Achieves up to 0.981 mAP50 with fine-tuning

Reaches 0.597 mAP50 on unseen species across datasets

Captures over 95% of species exploring only 33% of images

Abstract

We introduce OpenWildlife (OW), an open-vocabulary wildlife detector designed for multi-species identification in diverse aerial imagery. While existing automated methods perform well in specific settings, they often struggle to generalize across different species and environments due to limited taxonomic coverage and rigid model architectures. In contrast, OW leverages language-aware embeddings and a novel adaptation of the Grounding-DINO framework, enabling it to identify species specified through natural language inputs across both terrestrial and marine environments. Trained on 15 datasets, OW outperforms most existing methods, achieving up to \textbf{0.981} mAP50 with fine-tuning and \textbf{0.597} mAP50 on seven datasets featuring novel species. Additionally, we introduce an efficient search algorithm that combines k-nearest neighbors and breadth-first search to prioritize areas where social species are likely to be found. This approach captures over \textbf{95\%} of species while exploring only \textbf{33\%} of the available images. To support reproducibility, we publicly release our source code and dataset splits, establishing OW as a flexible, cost-effective solution for global biodiversity assessments.