Scientific Discovery by Generating Counterfactuals using Image Translation

TL;DR

This paper explores using explanation techniques and generative models to facilitate scientific discovery by generating and testing hypotheses from image classification models, specifically applied to retinal images predicting DME.

Contribution

It introduces a framework converting explanation outputs into discovery mechanisms and demonstrates how generative models can generate hypotheses without human priors.

Findings

Framework effectively explains the scientific mechanisms behind model predictions.

Generative models can produce hypotheses that align with known scientific features.

Application to retinal images reveals potential for scientific insights from AI models.

Abstract

Model explanation techniques play a critical role in understanding the source of a model's performance and making its decisions transparent. Here we investigate if explanation techniques can also be used as a mechanism for scientific discovery. We make three contributions: first, we propose a framework to convert predictions from explanation techniques to a mechanism of discovery. Second, we show how generative models in combination with black-box predictors can be used to generate hypotheses (without human priors) that can be critically examined. Third, with these techniques we study classification models for retinal images predicting Diabetic Macular Edema (DME), where recent work showed that a CNN trained on these images is likely learning novel features in the image. We demonstrate that the proposed framework is able to explain the underlying scientific mechanism, thus bridging the gap between the model's performance and human understanding.