Personal KPIs in IVF Laboratory: Are They Measurable or Distortable? A Case Study Using AI-Based Benchmarking

TL;DR

This study uses AI to assess whether personal KPIs in IVF labs are fair or biased based on patient and physician factors.

Contribution

The study introduces AI-based benchmarking to evaluate personal KPIs in IVF labs, adjusting for patient and physician variability.

Findings

Overall CPRs were similar between observed and predicted outcomes (0.31 vs. 0.33).

Age-stratified analysis showed significant discrepancies in the >40 group and excess CPR in the 35–40 group.

Physician-level comparisons suggested variability, but grouped calibration was not statistically significant.

Abstract

Background: Key performance indicators (KPIs) are widely used to evaluate embryologist performance in IVF laboratories, yet they are sensitive to patient demographics, treatment indications, and case allocation. Artificial intelligence (AI) offers opportunities to benchmark personal KPIs against context-aware expectations. This study evaluated whether personal CPR-based KPIs are measurable or distorted when compared with AI-derived predictions. Methods: We retrospectively analyzed 474 ICSI-only cycles performed by a single senior embryologist between 2022 and 2024. A Random Forest trained on 1294 institutional cycles generated AI-predicted clinical pregnancy rates (CPRs). Observed and predicted CPRs were compared across age groups, BMI categories, and physicians using cycle-level paired comparisons and a grouped calibration statistic. Results: Overall CPRs were similar between observed and predicted outcomes (0.31 vs. 0.33, p = 0.412). Age-stratified analysis showed significant discrepancy in the >40 group (0.11 vs. 0.18, p = 0.003), whereas CPR in the 35–40 group exceeded predictions (0.39 vs. 0.33, p = 0.018). BMI groups showed no miscalibration (p = 0.458). Physician-level comparisons suggested variability (p = 0.021), while grouped calibration was not statistically significant (p = 0.073). Conclusions: Personal embryologist KPIs are measurable but influenced by patient and physician factors. AI benchmarking may improve fairness by adjusting for case mix, yet systematic bias can persist in high-risk subgroups. Multi-operator, multi-center validation is needed to confirm generalizability.

Figures