# Dual-Model Weight Selection and Self-Knowledge Distillation for Medical Image Classification

## TL;DR

This paper introduces a novel lightweight medical image classification approach combining dual-model weight selection with self-knowledge distillation, enabling efficient knowledge transfer and improved performance on various medical imaging datasets.

## Contribution

It proposes a new method that integrates dual-model weight selection with self-knowledge distillation to enhance lightweight model performance in medical imaging.

## Key findings

- Outperforms existing methods on chest X-ray, CT, and MRI datasets.
- Achieves comparable accuracy to larger models with reduced computational cost.
- Demonstrates robustness across multiple medical imaging modalities.

## Abstract

We propose a novel medical image classification method that integrates dual-model weight selection with self-knowledge distillation (SKD). In real-world medical settings, deploying large-scale models is often limited by computational resource constraints, which pose significant challenges for their practical implementation. Thus, developing lightweight models that achieve comparable performance to large-scale models while maintaining computational efficiency is crucial. To address this, we employ a dual-model weight selection strategy that initializes two lightweight models with weights derived from a large pretrained model, enabling effective knowledge transfer. Next, SKD is applied to these selected models, allowing the use of a broad range of initial weight configurations without imposing additional excessive computational cost, followed by fine-tuning for the target classification tasks. By combining dual-model weight selection with self-knowledge distillation, our method overcomes the limitations of conventional approaches, which often fail to retain critical information in compact models. Extensive experiments on publicly available datasets-chest X-ray images, lung computed tomography scans, and brain magnetic resonance imaging scans-demonstrate the superior performance and robustness of our approach compared to existing methods.