Detection and Identification of Penguins Using Appearance and Motion Features

TL;DR

This paper presents a novel framework combining appearance and motion features to improve penguin detection and identification in challenging environments, utilizing adapted YOLO and contrastive learning for better accuracy and individual tracking.

Contribution

It introduces a two-frame input adaptation of YOLO for detection and a tracklet-based contrastive learning method for identification, enhancing performance in complex scenarios.

Findings

Detection mAP@0.5 improved from 0.922 to 0.933 with two-frame input

Motion cues help detect obscured targets better than static features

Feature embeddings show coherent clustering for same individuals

Abstract

In animal facilities, continuous surveillance of penguins is essential yet technically challenging due to their homogeneous visual characteristics, rapid and frequent posture changes, and substantial environmental noise such as water reflections. In this study, we propose a framework that enhances both detection and identification performance by integrating appearance and motion features. For detection, we adapted YOLO11 to process consecutive frames to overcome the lack of temporal consistency in single-frame detectors. This approach leverages motion cues to detect targets even when distinct visual features are obscured. Our evaluation shows that fine-tuning the model with two-frame inputs improves mAP@0.5 from 0.922 to 0.933, outperforming the baseline, and successfully recovers individuals that are indistinguishable in static images. For identification, we introduce a tracklet-based contrastive learning approach applied after tracking. Through qualitative visualization, we demonstrate that the method produces coherent feature embeddings, bringing samples from the same individual closer in the feature space, suggesting the potential for mitigating ID switching.