Parameter Efficient Fine-tuning for Domain-specific Gastrointestinal Disease Recognition

TL;DR

This paper introduces a parameter-efficient fine-tuning method using LoRA modules for domain-specific gastrointestinal disease recognition, outperforming traditional fine-tuning in accuracy and efficiency.

Contribution

The study proposes a low-rank adaptation approach for fine-tuning pretrained models, reducing parameters and improving performance in gastrointestinal disease classification.

Findings

LoRA-based fine-tuning outperforms end-to-end methods.

Significant parameter savings achieved with LoRA.

Enhanced accuracy in gastrointestinal disease recognition.

Abstract

Despite recent advancements in the field of medical image analysis with the use of pretrained foundation models, the issue of distribution shifts between cross-source images largely remains adamant. To circumvent that issue, investigators generally train a separate model for each source. However, this method becomes expensive when we fully fine-tune pretrained large models for a single dataset, as we must store multiple copies of those models. Thus, in this work, we propose using a low-rank adaptation (LoRA) module for fine-tuning downstream classification tasks. LoRAs learn lightweight task-specific low-rank matrices that perturb pretrained weights to optimize those downstream tasks. For gastrointestinal tract diseases, they exhibit significantly better results than end-to-end finetuning with improved parameter efficiency. Code is available at: github.com/sanjay931/peft-gi-recognition.