Cross-modulated Few-shot Image Generation for Colorectal Tissue Classification

TL;DR

This paper introduces XM-GAN, a novel few-shot image generation method for colorectal tissues that produces diverse, high-quality images to augment training data, improving classification accuracy in scarce data scenarios.

Contribution

The paper presents the first approach to few-shot colorectal tissue image generation, featuring a controllable fusion block for locally consistent image synthesis.

Findings

Pathologists differentiate generated and real images only 55% of the time.

Generated images improve classification accuracy by 4.4%.

The method achieves high-quality, diverse tissue image synthesis.

Abstract

In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues. Our few-shot generation method, named XM-GAN, takes one base and a pair of reference tissue images as input and generates high-quality yet diverse images. Within our XM-GAN, a novel controllable fusion block densely aggregates local regions of reference images based on their similarity to those in the base image, resulting in locally consistent features. To the best of our knowledge, we are the first to investigate few-shot generation in colorectal tissue images. We evaluate our few-shot colorectral tissue image generation by performing extensive qualitative, quantitative and subject specialist (pathologist) based evaluations. Specifically, in specialist-based evaluation, pathologists could differentiate between our XM-GAN generated tissue images and real images only 55% time. Moreover, we utilize these generated images as data augmentation to address the few-shot tissue image classification task, achieving a gain of 4.4% in terms of mean accuracy over the vanilla few-shot classifier. Code: \url{https://github.com/VIROBO-15/XM-GAN}