Performance comparison of YOLO, Faster R-CNN, and HRNet architectures for bull sperm viability assessment

TL;DR

This study compares AI models for analyzing bull sperm viability, finding that YOLOv8 and YOLOv12 perform best and outpace manual methods in speed and accuracy.

Contribution

The study introduces a performance comparison of four AI models for bull sperm viability assessment using eosin/nigrosin staining.

Findings

YOLOv8 achieved the highest balanced accuracy (97.1%) and F1 score (0.978) in test set analysis.

AI-based analysis was 16.7 times faster than manual counting, with no significant difference in accuracy compared to expert evaluation.

YOLOv5 + HRNet showed poor performance, especially in detecting dead sperm due to low specificity.

Abstract

Sperm viability analysis is directly related to fertility rates, and eosin/nigrosin staining is one of the most essential methods for assessing sperm viability. Although this method is easy to apply, it is time-consuming and has a high error rate due to its high subjectivity. Flow cytometric analyses, an alternative to the eosin/nigrosin method that is highly reliable, are not accessible due to the need for specialized personnel and high costs. Therefore, in this study, four different artificial intelligence models (YOLOv8, YOLOv12, Faster R-CNN, and YOLOv5 + HRNet) were utilized to analyze eosin/nigrosin-stained sperm samples with high reliability, and their performance was compared. For this purpose, 15 different frozen bull sperm samples were purchased, thawed in a water bath at 37 °C, and smears were prepared using the eosin/nigrosin method. A total of 3,068 photographs were taken from the prepared smears: 2140 for model training, 512 for validation, and 416 for testing. In the test set analysis, the YOLOv8 model demonstrated the highest overall performance (balanced accuracy: 97.1%; F1 score: 0.978), followed by YOLOv12 (balanced accuracy: 94.1%; F1 score: 0.958). While Faster R-CNN delivered mid-level performance, the YOLOv5 + HRNet model showed a significant performance loss, particularly in dead sperm detection, due to low specificity. To measure the clinical usability of the trained models, another smear set was prepared and photographed. In the independent mock test, 281 spermatozoa from 36 images were evaluated by the trained models and compared with both the ground truth and an expert evaluator. The expert achieved a balanced accuracy of 98.8%, while YOLOv12 demonstrated the highest performance among the AI models (balanced accuracy: 91.3%, F1 score: 0.895). Statistical comparison using McNemar’s test revealed no significant difference between YOLOv8, YOLOv12, Faster R-CNN, and the reference evaluation (p > 0.05). On average, AI-based analysis was 16.7 times faster than manual counting. The results reinforce the possibility that artificial intelligence models, updated with future studies, could be used in sperm viability analysis.

Figures