SiLVi: Simple Interface for Labeling Video Interactions

TL;DR

SiLVi is an open-source tool that combines behavioral labeling and localization in videos, enabling detailed annotation of animal and human interactions for computer vision research.

Contribution

It introduces a unified software platform that supports both behavior annotation and localization, filling a gap in existing open-source tools.

Findings

Supports annotation of behaviors and interactions within videos.

Generates structured outputs for training computer vision models.

Facilitates analysis of social and individualized animal behaviors.

Abstract

Computer vision methods are increasingly used for the automated analysis of large volumes of video data collected through camera traps, drones, or direct observations of animals in the wild. While recent advances have focused primarily on detecting individual actions, much less work has addressed the detection and annotation of interactions -- a crucial aspect for understanding social and individualized animal behavior. Existing open-source annotation tools support either behavioral labeling without localization of individuals, or localization without the capacity to capture interactions. To bridge this gap, we present SiLVi, an open-source labeling software that integrates both functionalities. SiLVi enables researchers to annotate behaviors and interactions directly within video data, generating structured outputs suitable for training and validating computer vision models. By linking behavioral ecology with computer vision, SiLVi facilitates the development of automated approaches for fine-grained behavioral analyses. Although developed primarily in the context of animal behavior, SiLVi could be useful more broadly to annotate human interactions in other videos that require extracting dynamic scene graphs. The software, along with documentation and download instructions, is available at: https://silvi.eckerlab.org.