Concept-Centric Visual Turing Tests for Method Validation

TL;DR

This paper introduces a concept-centric evaluation framework for medical image classification methods, inspired by the Turing Test, to better understand how models interpret key clinical concepts beyond traditional metrics.

Contribution

It proposes a novel interpretability assessment using a Twenty Questions paradigm and probabilistic modeling to evaluate medical imaging methods' understanding of clinical concepts.

Findings

The probabilistic model reveals dataset and method biases.

The approach reduces the number of queries needed for evaluation.

It provides deeper insights into model interpretability in clinical tasks.

Abstract

Recent advances in machine learning for medical imaging have led to impressive increases in model complexity and overall capabilities. However, the ability to discern the precise information a machine learning method is using to make decisions has lagged behind and it is often unclear how these performances are in fact achieved. Conventional evaluation metrics that reduce method performance to a single number or a curve only provide limited insights. Yet, systems used in clinical practice demand thorough validation that such crude characterizations miss. To this end, we present a framework to evaluate classification methods based on a number of interpretable concepts that are crucial for a clinical task. Our approach is inspired by the Turing Test concept and how to devise a test that adaptively questions a method for its ability to interpret medical images. To do this, we make use of a Twenty Questions paradigm whereby we use a probabilistic model to characterize the method's capacity to grasp task-specific concepts, and we introduce a strategy to sequentially query the method according to its previous answers. The results show that the probabilistic model is able to expose both the dataset's and the method's biases, and can be used to reduced the number of queries needed for confident performance evaluation.

Equations

$$f_c^{\mathcal{G}\mathcal{P}}(a_c)\sim \mathcal{G}\mathcal{P}\left({\mu }_c(a_c),k_c(a_c,a_c^{\prime })\right),$$

$$a_c=\frac{f(q)+q_{gt}}{2},$$

$$u_c=4{\int }_0^1{k}_c(a,a)da,$$

$$q^{\ast }\in \left\{q\in \mathcal{Q}:c_q\in \underset{c\in \mathcal{C}}{\mathrm{argmax}}(\max (u_c^{-},u_c^{+}))\text{ and }{q}_{gt}=o_{c_q}\right\},$$

Full text

1 Introduction

The field of medical image computing (MIC) has radically changed with the emergence of large neural networks, or Deep Learning (DL). For MIC tasks that were long considered extremely challenging, such as image-based pathology classification and segmentation, DL methods have now reached human-level performances on a variety of benchmarks.

Yet, as these methods have become increasingly powerful, the overall methodology to validate them has largely remained intact. For instance, challenge competitions compare different methods on a common dataset by using metrics most often borrowed from the computer vision literature. As recently noted in [9], challenge competition rankings and outcomes are very often highly skewed to the dataset or metrics used, and rarely relate to the clinical task. To tackle this, recent developments in visual question answering (VQA) [1, 5, 8, 14] methods, which answer questions related to image content, show the ability to infer concepts beyond traditional classification. Here again, however, the metrics used to evaluate VQA’s remain inadequate.

Instead, we consider an alternative approach to validating MIC methods, one inspired by Alan Turing’s Turing Test concept [13], where a human unknowingly communicates either with another human or an Artificial Intelligence system that produces answers. The aim of the test is to distinguish between the two based on a set of asked questions. Turing tests have been used in medical imaging to evaluate the quality of adversarial attacks, by seeing if an expert can distinguish between a real and an adversarial example [3, 12]. Another approach, focusing on the interpretability of methods that infer semantic information from images (e.g. classification, segmentation etc.) is seen in the work of Geman et al. [4] on automated Visual Turing Tests (VTT). In their work, an algorithm adaptively selects images and questions to pose to a method under evaluation (MuE) such that the answers can not be predicted from the history of answers. While this approach has increased explanatory power, it is limited to manually fabricated story lines to guide questioning. This makes it ill suited for medical applications where such story lines are hard to formalize.

For this reason, we propose a novel VTT framework to evaluate MIC methods (see Fig. 1). In particular, our approach focuses on evaluating MIC classification methods and we present a framework that tests if the method has correctly understood the relevant medical concepts when inferring test data. We do this by formulating our problem as a Twenty Questions game [2, 7] in which we model the likelihood of a given MuE to provide correct answers for different concepts present in test images. Our framework then sequentially picks test images and concepts such that the uncertainty of this model is reduced as quickly as possible. We demonstrate our framework in the context of three different multi-label classification problems where each concept is encoded by a given binary label.

2 Method

Our proposed VTT framework evaluates how a MuE, a MIC classification method in this case, performs with respect to core concepts relevant to the task for which it was trained. To do this, we make use of a validation image dataset that the MuE has never had access to, $\mathcal{D}=\{\mathbf{s}_{i}\}_{i=1}^{N_{S}}$, where $\mathbf{s}_{i}$ is an available test sample, such as an image or an arbitrary region within an image. As such, $N_{S}$ can be excessively large, as potentially millions of regions can be extracted from a single test image. For each $\mathbf{s}_{i}$, we denote the potential concepts that could be present in the sample as, $\mathcal{C}=\{c_{j}\}_{j=1}^{N_{C}}$. From this, we define a “question”, $q=(\mathbf{s}_{q},c_{q})\in\mathcal{D}\times\mathcal{C}=\mathcal{Q}$, of the form “Is concept $c_{q}$ present in sample $\mathbf{s}_{q}$?”. We let $q_{gt}\in\{0,1\}$ be the true answer to question $q$. In this work, we consider MuEs that perform multi-label classification tasks, $f:\mathcal{Q}\rightarrow[0,1]$, taking as input the question $q$ and producing the probability of the answer being “Yes”(see Fig. 1).

Given that evaluating all elements of $\mathcal{Q}$ may be computationally intractable, our VTT framework instead only evaluates a subset of $\mathcal{Q}$. We do this iteratively and adaptively, where we use the history of previously asked questions and their answers to build a performance model (Sec. 2.1). We then use a questioning strategy to determine which element of $\mathcal{Q}$ should be asked to the MuE. In particular, we propose a novel strategy that selects the element that maximally reduces the uncertainty in the performance model (Sec. 2.2). The process terminates after a fixed number of questions have been asked or when the uncertainty in the model has been reduced to an acceptable level. Fig. 2 (Left) illustrates our framework and we detail our performance model and questioning strategy next.

2.1 Performance Model

From a set of questions and the corresponding MuE responses, we aim to model the MuE performance with respect to concepts $\mathcal{C}$. While the relation between concepts could in practice be complex, we model them as independent here.

By its definition, the MuE provides answers to binary questions, and for any concept, there are 4 possible outcomes to a question: a True Negative (TN), a False Positive (FP), a False Negative (FN) or a True Positive (TP).

Our goal then is to model the relation between the frequency of these outcomes and the inputs to the MuE. To do this, we define a discrete random variable $Y_{c}$ for every concept $c\in\mathcal{C}$ which encodes the counts111The approach would be unchanged if the outcome frequency was used. of outcomes given by $f$. We achieve this by means of a Gaussian Process (GP) [11] of the form,

[TABLE]

where $\mu_{c}(\cdot)$ and $k_{c}(\cdot,\cdot)$ are the mean and kernel functions of the GP, respectively, and

[TABLE]

describes the answer of $f$ with respect to the question $q$. The mean function $\mu_{c}$ is initialized to [math], and the covariance function (kernel) $k_{c}$ is a squared exponential $k_{c}(a_{c,m},a_{c,n})=\sigma_{f}\cdot e^{-\frac{1}{2l^{2}}(a_{c,m}-a_{c,n})^{2}}+\sigma_{n}\delta_{mn}$, with characteristic length-scale $l=0.1$, initial signal variance $\sigma_{f}=1$ and noise variance $\sigma_{n}=0.025$.

To then infer the value of $Y_{c}$ for any $a_{c}$, we store the pairs $\{(a^{(i)}_{c},y_{c}^{(i)})\}$ where $y_{c}^{(i)}$ is the number of times $f$ has given $a=a^{(i)}_{c}$, and use these as observations to infer the complete model $f_{c}^{\mathcal{GP}}$ using standard inference [11]. In practice, we discretized the range of $a$ in bins of $\Delta a=0.01$.

A consequence of this model is that we can now visualize the performance of $f$ with respect to the concepts in $\mathcal{C}$, the selected $q$’s and the dataset $\mathcal{D}$. In Fig. 2 (Right), we illustrate $f^{\mathcal{GP}}_{c}$ in terms of $a$ for each $c$. Such a visualization depicts any bias that both $f$ and the dataset $\mathcal{D}$ may have (e. g., $\mathcal{D}$ contains few samples regarding a specific concept). Note that by summing up the observations over concepts or integrating over the four different subregions of the support set, one retrieves the total TN, FP, FN and TP, and other subsequent metrics.

2.2 Questioning Strategy

With the performance model above and the fact that the validation dataset $\mathcal{D}$ may be intractably larger, we now describe how to select samples from $\mathcal{Q}$ to verify that the MuE has grasped relevant concepts.

To do this, we present a strategy that looks to select samples $\mathbf{s}$ and concepts $c$ that are likely to reduce our model’s uncertainty. We do this by computing the uncertainty of a concept as the integral of the $95\%$ confidence region of the GP over its support set (i. e., 2 standard deviations). That is, for concept $c\in C$,

[TABLE]

which can be decomposed as $u_{c}^{-}=4\int_{0}^{0.5}{k_{c}(a,a)da}$ and $u_{c}^{+}=4\int_{0.5}^{1}{k_{c}(a,a)da}$, corresponding to the uncertainty associated with negative and positive samples in $\mathcal{D}$, respectively. Visually this corresponds to the area over the intervals $a=[0,0.5]$ and $a=[0.5,1]$ (see Fig. 2, Right).

Our strategy then chooses which concept to ask, and whether to ask a sample that has or does not have this concept in it. This is performed by selecting

[TABLE]

where $o_{c_{q}}=0$ if $\max(u_{c}^{-},u_{c}^{+})=u_{c}^{-}$ or 1 if $\max(u_{c}^{-},u_{c}^{+})=u_{c}^{+}$.

From this, $q^{*}$ is selected either randomly, or based on its unpredictability. As in [4], the latter is computed using the same dataset $\mathcal{D}$ and the history of already answered questions $\mathcal{H}$. An overview of the questioning strategy can be seen in Algorithm 1.

3 Experiments and Results

We choose to evaluate our framework on multi-label classification tasks where concepts are directly linked to groundtruth labels. In the following, we outline the different datasets and MuEs we use, and our experimental setup.

3.1 Datasets and MuE

Indian Diabetic Retinopathy image Dataset (IDRiD) [10]: 143 fundus images from both healthy and diabetic retinopathy subjects, with the task of identifying 4 different lesion types (hemorrhage, hard and soft exudates, microaneurysm). The multilabel MuE is a ResNet [6] with pre-trained weights, trained with 100 images. The remaining 43 images were used as samples in $\mathcal{D}$.

ISIC 2018 Skin Lesion Analysis222https://challenge2018.isic-archive.com/: 1’876 dermoscopic images of skin lesions. Here $N_{C}=5$, consisting of different skin lesions types: pigment network, negative network, globules, milia like cysts and streaks. The MuE is a pre-trained ResNet [6] trained with 70% of data. A hundred randomly chosen images from the 30% test images were used to populate $\mathcal{D}$.

OCT: 200 OCT cross-sectional slices from Age-Related Macular Degeneration and Diabetic Macular Edema patients. Eleven different biomarkers potentially can be present in any cross-section ($N_{C}=11$). All 200 images are used for $\mathcal{D}$ and a seperate trained Dilated Residual Network [15] is used for the MuE.

Given that we are not focused on optimizing the performance of a specific MuE but rather on evaluating relative behavior with respect to concepts, we also provide a set of synthetically generated MuEs for which we understand their performance fully. That is, given that the distribution of concepts in $\mathcal{D}$ is not uniform for any of the datasets, we wish to compare each MuE to a “biased” MuE. To do this, we simulate a biased MuE that answers with $90\%$ accuracy questions regarding the most common concept and $50\%$ on all others. Similarly, we simulate a MuE with a $50\%$ accuracy for the most common concept, and $90\%$ on all others. Last, we also simulate an unbiased algorithm, with a $70\%$ accuracy regardless of the concept or class imbalance.

3.2 Experiments

To evaluate our framework, we compare four questioning strategies: (a) random, (b) based on the predictability of the question given the previous questions and the dataset [4], (c) based on the uncertainty of the question as defined in Sec. 2.2, and (d) based on the combination of the above two, as described in Sec. 2.2.

Fig. 3 depicts the state of the performance for two concepts (GA and Drusen) after a hundred questions have been asked on the OCT dataset using the trained MuE. In all plots the bias in concept occurrence can be observed and we see that both the random strategy, and the one relying solely on the unpredictability criteria, are prone to not ask many questions about rare concepts (GA concept), thus delaying the assessment of the MuE’s on this concept. Both uncertainty based methods however manage to sample adequately and are more confident in the performance model.

In another experiment, we ask all possible questions following each one of the four described strategies. That is, the state of the performance model is the same after all questions have been asked. We repeat the experiments 10 times, and monitor the average uncertainty as questions are asked. The results are shown in Fig. 4. Here we observe that the proposed questioning strategies reach saturated levels of uncertainty quicker than the other strategies, and would require fewer questions for confident assessments.

4 Discussion

To summarize, we present a more informative and interpretable performance model for evaluating closed-end, “Yes/No”  inference methods. To this, we propose a strategy to sample from the - possibly intractable - set of questions, in order to reach high certainty in the performance model characterizing the MuE and the validation data. We assess our method on three different medical imaging datasets and show that the performance model is able to capture the data distribution information and the MuE biases. Moreover, the questioning strategy allows for faster convergence of the performance model to a low uncertainty state. We will look to extend this concept to segmentation problems in future work.