Comparison of attention models and post-hoc explanation methods for embryo stage identification: a case study

TL;DR

This study evaluates attention and post-hoc explanation methods for embryo stage identification, revealing inconsistencies in faithfulness metrics and highlighting challenges in interpreting AI models in IVF.

Contribution

It benchmarks explanation methods using objective faithfulness metrics and analyzes their reliability and biases in embryo stage classification.

Findings

Metrics show low agreement on model ranking

Post-hoc and attention methods are favored depending on the metric

Highlights challenges in defining faithfulness for explanations

Abstract

An important limitation to the development of AI-based solutions for In Vitro Fertilization (IVF) is the black-box nature of most state-of-the-art models, due to the complexity of deep learning architectures, which raises potential bias and fairness issues. The need for interpretable AI has risen not only in the IVF field but also in the deep learning community in general. This has started a trend in literature where authors focus on designing objective metrics to evaluate generic explanation methods. In this paper, we study the behavior of recently proposed objective faithfulness metrics applied to the problem of embryo stage identification. We benchmark attention models and post-hoc methods using metrics and further show empirically that (1) the metrics produce low overall agreement on the model ranking and (2) depending on the metric approach, either post-hoc methods or attention models are favored. We conclude with general remarks about the difficulty of defining faithfulness and the necessity of understanding its relationship with the type of approach that is favored.