# AI–Driven Multimodal Sensing for Early Detection of Health Disorders in Dairy Cows

## TL;DR

This paper introduces an AI system that uses multiple data sources to detect early health issues in dairy cows, improving accuracy compared to single-source methods.

## Contribution

The key novelty is a hybrid deep learning model combining U-Net, O-Net, and ResNet for multimodal health monitoring in dairy cows.

## Key findings

- The system achieved 91.62% accuracy and 0.94 AUC in detecting health deviations in dairy cows.
- Multimodal data integration improved classification performance by up to 3% compared to single-modality models.
- The framework enables early detection of udder, leg, and hoof infections before clinical symptoms appear.

## Abstract

This manuscript presents a multi-modal artificial intelligence framework for health status and welfare detection of dairy cows. Data from the milking system, internal sensing devices, and thermal cameras were jointly analyzed to enable comprehensive health monitoring. All measurements, except the internal sensor, are collected in a contactless manner, avoiding direct physical interaction with the animals. By linking changes in production and physiological data with visual temperature patterns, the system can identify health warning signs of disease more reliably than when using a single data source. The system incorporates a novel hybrid deep learning architecture that unifies the backbone structures of U-Net, O-Net, and ResNet, facilitating multi-scale feature learning for accurate analysis of dairy cow health conditions. This combined approach demonstrates improved discrimination performance in retrospective analysis and indicates potential for earlier identification of health-related deviations, which may support the development of decision-support tools for dairy cow health management.

Digital technologies that continuously quantify animal behavior, physiology, and production offer significant potential for the early identification of health and welfare disorders of dairy cows. In this study, a multimodal artificial intelligence (AI) framework is proposed for real-time health monitoring of dairy cows through the integration of physiological, behavioral, production, and thermal imaging data, targeting veterinarian-confirmed udder, leg, and hoof infections. Predictions are generated at the cow-day level by aggregating multimodal measurements collected during daily milking events. The dataset comprised 88 lactating cows, including veterinarian-confirmed udder, leg, and hoof infections grouped under a single ‘sick’ label. To prevent information leakage, model evaluation was performed using a cow-level data split, ensuring that data from the same animal did not appear in both training and testing sets. The system is designed to detect early deviations from normal health trajectories prior to the appearance of overt clinical symptoms. All measurements, with the exception of the intra-ruminal bolus sensor, were obtained non-invasively within a commercial dairy farm equipped with automated milking and monitoring infrastructure. A key novelty of this work is the simultaneous integration of data from three independent sources: an automated milking system, a thermal imaging camera, and an intra-ruminal bolus sensor. A hybrid deep learning architecture is introduced that combines the core components of established models, including U-Net, O-Net, and ResNet, to exploit their complementary strengths for the analysis of dairy cow health states. The proposed multimodal approach achieved an overall accuracy of 91.62% and an AUC of 0.94 and improved classification performance by up to 3% compared with single-modality models, demonstrating enhanced robustness and sensitivity to early-stage disease.