AnimalClue: Recognizing Animals by their Traces

TL;DR

AnimalClue introduces a large-scale dataset for identifying animal species from indirect evidence like footprints and feces, addressing a significant gap in wildlife monitoring using computer vision.

Contribution

It provides the first extensive dataset for species identification from indirect clues, enabling new research in wildlife monitoring and computer vision.

Findings

Existing models face challenges in recognizing subtle features in traces.

The dataset enables benchmarking of detection, classification, and segmentation tasks.

Key challenges include fine-grained recognition and environmental variability.

Abstract

Wildlife observation plays an important role in biodiversity conservation, necessitating robust methodologies for monitoring wildlife populations and interspecies interactions. Recent advances in computer vision have significantly contributed to automating fundamental wildlife observation tasks, such as animal detection and species identification. However, accurately identifying species from indirect evidence like footprints and feces remains relatively underexplored, despite its importance in contributing to wildlife monitoring. To bridge this gap, we introduce AnimalClue, the first large-scale dataset for species identification from images of indirect evidence. Our dataset consists of 159,605 bounding boxes encompassing five categories of indirect clues: footprints, feces, eggs, bones, and feathers. It covers 968 species, 200 families, and 65 orders. Each image is annotated with species-level labels, bounding boxes or segmentation masks, and fine-grained trait information, including activity patterns and habitat preferences. Unlike existing datasets primarily focused on direct visual features (e.g., animal appearances), AnimalClue presents unique challenges for classification, detection, and instance segmentation tasks due to the need for recognizing more detailed and subtle visual features. In our experiments, we extensively evaluate representative vision models and identify key challenges in animal identification from their traces. Our dataset and code are available at https://dahlian00.github.io/AnimalCluePage/