Multi-view Tracking, Re-ID, and Social Network Analysis of a Flock of Visually Similar Birds in an Outdoor Aviary

TL;DR

This paper presents a novel multi-view tracking system for analyzing social interactions of birds in a 3D aviary, addressing challenges like occlusions and orientation, and introduces a new dataset for evaluation.

Contribution

It introduces a new system for 3D multi-view tracking of birds, along with a dataset and analysis of social behaviors influenced by context.

Findings

Social context influences interaction sequences among birds.

The system effectively tracks birds despite occlusions and orientation challenges.

A new dataset enables evaluation of multi-view tracking in complex environments.

Abstract

The ability to capture detailed interactions among individuals in a social group is foundational to our study of animal behavior and neuroscience. Recent advances in deep learning and computer vision are driving rapid progress in methods that can record the actions and interactions of multiple individuals simultaneously. Many social species, such as birds, however, live deeply embedded in a three-dimensional world. This world introduces additional perceptual challenges such as occlusions, orientation-dependent appearance, large variation in apparent size, and poor sensor coverage for 3D reconstruction, that are not encountered by applications studying animals that move and interact only on 2D planes. Here we introduce a system for studying the behavioral dynamics of a group of songbirds as they move throughout a 3D aviary. We study the complexities that arise when tracking a group of closely interacting animals in three dimensions and introduce a novel dataset for evaluating multi-view trackers. Finally, we analyze captured ethogram data and demonstrate that social context affects the distribution of sequential interactions between birds in the aviary.