Interaction-via-Actions: Cattle Interaction Detection with Joint Learning of Action-Interaction Latent Space

TL;DR

This paper presents CattleAct, a data-efficient method for detecting cattle interactions from images by learning a joint action-interaction latent space, enabling accurate behavior analysis for livestock management.

Contribution

It introduces a novel approach combining action latent space learning with contrastive fine-tuning to detect rare cattle interactions from limited data.

Findings

Accurate interaction detection on commercial pasture data

Effective embedding of rare interactions via contrastive learning

System integration of video and GPS for real-time monitoring

Abstract

This paper introduces a method and application for automatically detecting behavioral interactions between grazing cattle from a single image, which is essential for smart livestock management in the cattle industry, such as for detecting estrus. Although interaction detection for humans has been actively studied, a non-trivial challenge lies in cattle interaction detection, specifically the lack of a comprehensive behavioral dataset that includes interactions, as the interactions of grazing cattle are rare events. We, therefore, propose CattleAct, a data-efficient method for interaction detection by decomposing interactions into the combinations of actions by individual cattle. Specifically, we first learn an action latent space from a large-scale cattle action dataset. Then, we embed rare interactions via the fine-tuning of the pre-trained latent space using contrastive learning, thereby constructing a unified latent space of actions and interactions. On top of the proposed method, we develop a practical working system integrating video and GPS inputs. Experiments on a commercial-scale pasture demonstrate the accurate interaction detection achieved by our method compared to the baselines. Our implementation is available at https://github.com/rakawanegan/CattleAct.