Lameness detection in dairy cows using pose estimation and bidirectional LSTMs

TL;DR

This paper introduces a novel, markerless approach combining pose estimation and bidirectional LSTMs to detect lameness in dairy cows, achieving high accuracy with minimal video data and small datasets.

Contribution

The study presents a new method that integrates pose estimation with BLSTM neural networks for efficient, accurate, and data-efficient lameness detection in dairy cows.

Findings

Achieved 85% classification accuracy, outperforming traditional feature-based methods.

Detected lameness with as little as one second of video data.

Eliminated manual feature engineering through deep learning.

Abstract

This study presents a lameness detection approach that combines pose estimation and Bidirectional Long-Short-Term Memory (BLSTM) neural networks. Combining pose-estimation and BLSTMs classifier offers the following advantages: markerless pose-estimation, elimination of manual feature engineering by learning temporal motion features from the keypoint trajectories, and working with short sequences and small training datasets. Motion sequences of nine keypoints (located on the cows' hooves, head and back) were extracted from videos of walking cows with the T-LEAP pose estimation model. The trajectories of the keypoints were then used as an input to a BLSTM classifier that was trained to perform binary lameness classification. Our method significantly outperformed an established method that relied on manually-designed locomotion features: our best architecture achieved a classification accuracy of 85%, against 80% accuracy for the feature-based approach. Furthermore, we showed that our BLSTM classifier could detect lameness with as little as one second of video data.