Shuffle Instances-based Vision Transformer for Pancreatic Cancer ROSE Image Classification

TL;DR

This paper introduces a shuffle instances-based Vision Transformer (SI-ViT) that improves pancreatic cancer image classification by reducing perturbations and focusing on relevant cellular patterns, achieving higher accuracy in ROSE image analysis.

Contribution

The novel SI-ViT approach effectively handles perturbations and enhances instance modeling in cytopathological image classification, outperforming existing methods.

Findings

Significant accuracy improvements in pancreatic cancer classification.

Enhanced attention to relevant cellular regions.

Effective reduction of staining and device-related perturbations.

Abstract

The rapid on-site evaluation (ROSE) technique can signifi-cantly accelerate the diagnosis of pancreatic cancer by im-mediately analyzing the fast-stained cytopathological images. Computer-aided diagnosis (CAD) can potentially address the shortage of pathologists in ROSE. However, the cancerous patterns vary significantly between different samples, making the CAD task extremely challenging. Besides, the ROSE images have complicated perturbations regarding color distribution, brightness, and contrast due to different staining qualities and various acquisition device types. To address these challenges, we proposed a shuffle instances-based Vision Transformer (SI-ViT) approach, which can reduce the perturbations and enhance the modeling among the instances. With the regrouped bags of shuffle instances and their bag-level soft labels, the approach utilizes a regression head to make the model focus on the cells rather than various perturbations. Simultaneously, combined with a classification head, the model can effectively identify the general distributive patterns among different instances. The results demonstrate significant improvements in the classification accuracy with more accurate attention regions, indicating that the diverse patterns of ROSE images are effectively extracted, and the complicated perturbations are significantly reduced. It also suggests that the SI-ViT has excellent potential in analyzing cytopathological images. The code and experimental results are available at https://github.com/sagizty/MIL-SI.