CattleFace-RGBT: RGB-T Cattle Facial Landmark Benchmark

TL;DR

This paper introduces CattleFace-RGBT, a novel RGB-T cattle facial landmark dataset with 2,300 image pairs, and benchmarks various models to facilitate future research in cattle health monitoring using thermal and RGB imaging.

Contribution

The paper presents the first RGB-T cattle facial landmark dataset and a semi-automatic annotation method leveraging transfer learning and AI-assisted annotation tools.

Findings

Benchmark results for various backbone architectures

Effective transfer learning from RGB to thermal images

Potential for improved cattle health assessment

Abstract

To address this challenge, we introduce CattleFace-RGBT, a RGB-T Cattle Facial Landmark dataset consisting of 2,300 RGB-T image pairs, a total of 4,600 images. Creating a landmark dataset is time-consuming, but AI-assisted annotation can help. However, applying AI to thermal images is challenging due to suboptimal results from direct thermal training and infeasible RGB-thermal alignment due to different camera views. Therefore, we opt to transfer models trained on RGB to thermal images and refine them using our AI-assisted annotation tool following a semi-automatic annotation approach. Accurately localizing facial key points on both RGB and thermal images enables us to not only discern the cattle's respiratory signs but also measure temperatures to assess the animal's thermal state. To the best of our knowledge, this is the first dataset for the cattle facial landmark on RGB-T images. We conduct benchmarking of the CattleFace-RGBT dataset across various backbone architectures, with the objective of establishing baselines for future research, analysis, and comparison. The dataset and models are at https://github.com/UARK-AICV/CattleFace-RGBT-benchmark