Skewness-Guided Pruning of Multimodal Swin Transformers for Federated Skin Lesion Classification on Edge Devices

TL;DR

This paper introduces a skewness-guided pruning technique for multimodal Swin Transformers in federated skin lesion classification, significantly reducing model size while preserving accuracy for edge device deployment.

Contribution

It presents a novel pruning method based on statistical skewness to optimize multimodal Swin Transformers for federated learning in medical imaging.

Findings

36% reduction in model size without accuracy loss

Effective model compression for privacy-preserving edge deployment

Validated in federated learning environment

Abstract

In recent years, high-performance computer vision models have achieved remarkable success in medical imaging, with some skin lesion classification systems even surpassing dermatology specialists in diagnostic accuracy. However, such models are computationally intensive and large in size, making them unsuitable for deployment on edge devices. In addition, strict privacy constraints hinder centralized data management, motivating the adoption of Federated Learning (FL). To address these challenges, this study proposes a skewness-guided pruning method that selectively prunes the Multi-Head Self-Attention and Multi-Layer Perceptron layers of a multimodal Swin Transformer based on the statistical skewness of their output distributions. The proposed method was validated in a horizontal FL environment and shown to maintain performance while substantially reducing model complexity. Experiments on the compact Swin Transformer demonstrate approximately 36\% model size reduction with no loss in accuracy. These findings highlight the feasibility of achieving efficient model compression and privacy-preserving distributed learning for multimodal medical AI on edge devices.