S-E Pipeline: A Vision Transformer (ViT) based Resilient Classification Pipeline for Medical Imaging Against Adversarial Attacks

TL;DR

This paper introduces S-E Pipeline, a preprocessing method combining segmentation and enhancement techniques to improve Vision Transformer's resilience against adversarial attacks in medical imaging, with significant attack mitigation demonstrated.

Contribution

The paper presents a novel preprocessing pipeline that enhances ViT robustness against adversarial attacks using segmentation and image enhancement techniques.

Findings

Reduces adversarial attack impact by over 70% on ViT models.

Effective in resource-constrained environments like NVIDIA Jetson Orin Nano.

Improves critical feature preservation in medical image classification.

Abstract

Vision Transformer (ViT) is becoming widely popular in automating accurate disease diagnosis in medical imaging owing to its robust self-attention mechanism. However, ViTs remain vulnerable to adversarial attacks that may thwart the diagnosis process by leading it to intentional misclassification of critical disease. In this paper, we propose a novel image classification pipeline, namely, S-E Pipeline, that performs multiple pre-processing steps that allow ViT to be trained on critical features so as to reduce the impact of input perturbations by adversaries. Our method uses a combination of segmentation and image enhancement techniques such as Contrast Limited Adaptive Histogram Equalization (CLAHE), Unsharp Masking (UM), and High-Frequency Emphasis filtering (HFE) as preprocessing steps to identify critical features that remain intact even after adversarial perturbations. The experimental study demonstrates that our novel pipeline helps in reducing the effect of adversarial attacks by 72.22% for the ViT-b32 model and 86.58% for the ViT-l32 model. Furthermore, we have shown an end-to-end deployment of our proposed method on the NVIDIA Jetson Orin Nano board to demonstrate its practical use case in modern hand-held devices that are usually resource-constrained.