# Investigating discrepancies in accuracy, agreement and interpretability for single-frame embryo classification tasks conducted by embryologists and deep learning models

## TL;DR

This study compares how well embryologists and AI models classify embryos, finding that while AI can be accurate, it lacks consistency and interpretability.

## Contribution

The paper introduces a framework evaluating AI in embryo classification by combining accuracy, agreement, and interpretability.

## Key findings

- Embryologists outperformed deep learning models in classification accuracy.
- ResNet-34 produced more biologically relevant explanations than VGG16.
- Interpretability did not consistently correlate with model accuracy.

## Abstract

Artificial intelligence tools show promise in supporting clinical decision making, but their safe use requires evaluation of not only accuracy, but also agreement with experts and interpretability of model decisions. The aim of this study was to evaluate the accuracy and agreement of human embryologists and deep learning models in embryo stage classification, and to explore interpretability through explainable artificial intelligence.

A retrospective, single-center study used single-frame embryo images (n = 245) classified according to developmental stage by three embryologists and two deep learning models, ResNet-34 and VGG16. Accuracy and agreement among all operators was evaluated, along with an assessment of interpretability with regards to model-generated explanations for spatial attention.

Embryologists achieved higher accuracy (89.9%) than ResNet-34 (78.8%, p < 0.001) and VGG16 (74.3%, p < 0.001), while overall agreement with the reference standard remained excellent for all operators (κ≥0.932). Stage-wise agreement was consistently stronger among embryologists than DL models (κ = 0.778–0.952 vs. 0.385–0.681). ResNet-34 Grad-CAMs were rated biologically relevant more often than VGG16 (89% vs. 59%, p < 0.001), yet interpretability did not consistently align with accuracy. Analysis of spatial overlap between model generated explanations was weak and observed to be lowest at the blastocyst stage, despite perfect model accuracy.

These findings highlight the need for evaluation frameworks that integrate accuracy, agreement and interpretability to support safe and transparent development of artificial intelligence tools in assisted reproduction technology.