# Distill-to-Label: Weakly Supervised Instance Labeling Using Knowledge Distillation

## TL;DR

This paper introduces a weakly supervised instance labeling method using knowledge distillation and virtual adversarial training to improve instance predictions from image-level labels, especially in medical imaging.

## Contribution

It repurposes MIL models for instance-level prediction with a novel regularization and distillation approach, addressing small datasets and non-local patterns.

## Key findings

- Outperforms existing MIL methods in cancer detection tasks
- Produces high-quality instance-level predictions
- Effective in histopathological image analysis

## Abstract

Weakly supervised instance labeling using only image-level labels, in lieu of expensive fine-grained pixel annotations, is crucial in several applications including medical image analysis. In contrast to conventional instance segmentation scenarios in computer vision, the problems that we consider are characterized by a small number of training images and non-local patterns that lead to the diagnosis. In this paper, we explore the use of multiple instance learning (MIL) to design an instance label generator under this weakly supervised setting. Motivated by the observation that an MIL model can handle bags of varying sizes, we propose to repurpose an MIL model originally trained for bag-level classification to produce reliable predictions for single instances, i.e., bags of size $1$. To this end, we introduce a novel regularization strategy based on virtual adversarial training for improving MIL training, and subsequently develop a knowledge distillation technique for repurposing the trained MIL model. Using empirical studies on colon cancer and breast cancer detection from histopathological images, we show that the proposed approach produces high-quality instance-level prediction and significantly outperforms state-of-the MIL methods.