Real-Time Cattle Interaction Recognition via Triple-stream Network

TL;DR

This paper presents a real-time cattle interaction recognition system using a triple-stream network that fuses visual, geometric, and semantic features, addressing the challenge of social behavior detection in stockbreeding.

Contribution

The novel triple-stream network effectively combines multiple features for real-time cattle interaction recognition, incorporating self-supervised pre-training to mitigate limited labeled data.

Findings

Achieves accurate real-time cattle interaction recognition

Outperforms existing methods in qualitative and quantitative evaluations

Demonstrates the effectiveness of multi-feature fusion in social behavior detection

Abstract

In stockbreeding of beef cattle, computer vision-based approaches have been widely employed to monitor cattle conditions (e.g. the physical, physiology, and health). To this end, the accurate and effective recognition of cattle action is a prerequisite. Generally, most existing models are confined to individual behavior that uses video-based methods to extract spatial-temporal features for recognizing the individual actions of each cattle. However, there is sociality among cattle and their interaction usually reflects important conditions, e.g. estrus, and also video-based method neglects the real-time capability of the model. Based on this, we tackle the challenging task of real-time recognizing interactions between cattle in a single frame in this paper. The pipeline of our method includes two main modules: Cattle Localization Network and Interaction Recognition Network. At every moment, cattle localization network outputs high-quality interaction proposals from every detected cattle and feeds them into the interaction recognition network with a triple-stream architecture. Such a triple-stream network allows us to fuse different features relevant to recognizing interactions. Specifically, the three kinds of features are a visual feature that extracts the appearance representation of interaction proposals, a geometric feature that reflects the spatial relationship between cattle, and a semantic feature that captures our prior knowledge of the relationship between the individual action and interaction of cattle. In addition, to solve the problem of insufficient quantity of labeled data, we pre-train the model based on self-supervised learning. Qualitative and quantitative evaluation evidences the performance of our framework as an effective method to recognize cattle interaction in real time.