Recurrence over Video Frames (RoVF) for the Re-identification of Meerkats

TL;DR

This paper introduces RoVF, a recurrent video-based embedding method for animal re-identification, demonstrating improved accuracy over existing transformer models on meerkat videos, aiding conservation efforts.

Contribution

The paper presents RoVF, a novel recurrent architecture using Perceiver for video-based animal re-identification, with training via triplet loss without individual IDs, outperforming transformer baselines.

Findings

RoVF achieves 49% top-1 accuracy, surpassing DINOv2's 42%.

RoVF can identify individuals humans cannot match.

Minimal fine-tuning yields better results than other models.

Abstract

Deep learning approaches for animal re-identification have had a major impact on conservation, significantly reducing the time required for many downstream tasks, such as well-being monitoring. We propose a method called Recurrence over Video Frames (RoVF), which uses a recurrent head based on the Perceiver architecture to iteratively construct an embedding from a video clip. RoVF is trained using triplet loss based on the co-occurrence of individuals in the video frames, where the individual IDs are unavailable. We tested this method and various models based on the DINOv2 transformer architecture on a dataset of meerkats collected at the Wellington Zoo. Our method achieves a top-1 re-identification accuracy of $49\%$, which is higher than that of the best DINOv2 model ($42\%$). We found that the model can match observations of individuals where humans cannot, and our model (RoVF) performs better than the comparisons with minimal fine-tuning. In future work, we plan to improve these models by using pre-text tasks, apply them to animal behaviour classification, and perform a hyperparameter search to optimise the models further.