TAFM-Net: A Novel Approach to Skin Lesion Segmentation Using Transformer Attention and Focal Modulation

TL;DR

TAFM-Net introduces a transformer-based, attention-enhanced skin lesion segmentation model that outperforms previous methods by integrating focal modulation and a dynamic loss function, demonstrating high accuracy on multiple datasets.

Contribution

The paper presents TAFM-Net, a novel skin lesion segmentation model combining transformer attention, focal modulation, and a dynamic loss function for improved performance and interpretability.

Findings

Achieved Jaccard scores of 93.64%, 86.88%, and 92.88% on ISIC datasets.

Enhanced segmentation accuracy over traditional U-Net models.

Demonstrated potential for real-world clinical application.

Abstract

Incorporating modern computer vision techniques into clinical protocols shows promise in improving skin lesion segmentation. The U-Net architecture has been a key model in this area, iteratively improved to address challenges arising from the heterogeneity of dermatologic images due to varying clinical settings, lighting, patient attributes, and hair density. To further improve skin lesion segmentation, we developed TAFM-Net, an innovative model leveraging self-adaptive transformer attention (TA) coupled with focal modulation (FM). Our model integrates an EfficientNetV2B1 encoder, which employs TA to enhance spatial and channel-related saliency, while a densely connected decoder integrates FM within skip connections, enhancing feature emphasis, segmentation performance, and interpretability crucial for medical image analysis. A novel dynamic loss function amalgamates region and boundary information, guiding effective model training. Our model achieves competitive performance, with Jaccard coefficients of 93.64\%, 86.88\% and 92.88\% in the ISIC2016, ISIC2017 and ISIC2018 datasets, respectively, demonstrating its potential in real-world scenarios.