More Than Positive and Negative: Communicating Fine Granularity in Medical Diagnosis

TL;DR

This paper introduces a new benchmark and metric for fine-grained classification in medical diagnosis, enabling AI systems to distinguish between subcategories within positive cases, thus capturing more detailed medical knowledge.

Contribution

It proposes a novel benchmark, a new evaluation metric, and a simple risk modulation method for fine granularity learning from medical images using coarse labels.

Findings

The new benchmark encourages detailed classification in medical AI.

The proposed risk modulation method outperforms existing approaches.

Empirical results demonstrate the effectiveness of the simple baseline.

Abstract

With the advance of deep learning, much progress has been made in building powerful artificial intelligence (AI) systems for automatic Chest X-ray (CXR) analysis. Most existing AI models are trained to be a binary classifier with the aim of distinguishing positive and negative cases. However, a large gap exists between the simple binary setting and complicated real-world medical scenarios. In this work, we reinvestigate the problem of automatic radiology diagnosis. We first observe that there is considerable diversity among cases within the positive class, which means simply classifying them as positive loses many important details. This motivates us to build AI models that can communicate fine-grained knowledge from medical images like human experts. To this end, we first propose a new benchmark on fine granularity learning from medical images. Specifically, we devise a division rule based on medical knowledge to divide positive cases into two subcategories, namely atypical positive and typical positive. Then, we propose a new metric termed AUC$^\text{FG}$ on the two subcategories for evaluation of the ability to separate them apart. With the proposed benchmark, we encourage the community to develop AI diagnosis systems that could better learn fine granularity from medical images. Last, we propose a simple risk modulation approach to this problem by only using coarse labels in training. Empirical results show that despite its simplicity, the proposed method achieves superior performance and thus serves as a strong baseline.