Regularized Low-Rank Adaptation for Few-Shot Organ Segmentation

TL;DR

This paper introduces a novel regularization technique for Low-Rank Adaptation in medical image segmentation that automatically adjusts the intrinsic rank during fine-tuning, leading to improved performance in few-shot settings.

Contribution

It proposes a dynamic rank adjustment method for LoRA using an l_1 sparsity regularizer, enhancing adaptability and performance in medical image segmentation tasks.

Findings

Significant performance improvements over standard LoRA.

Robustness against suboptimal rank initialization.

Effective in few-shot medical image segmentation scenarios.

Abstract

Parameter-efficient fine-tuning (PEFT) of pre-trained foundation models is increasingly attracting interest in medical imaging due to its effectiveness and computational efficiency. Among these methods, Low-Rank Adaptation (LoRA) is a notable approach based on the assumption that the adaptation inherently occurs in a low-dimensional subspace. While it has shown good performance, its implementation requires a fixed and unalterable rank, which might be challenging to select given the unique complexities and requirements of each medical imaging downstream task. Inspired by advancements in natural image processing, we introduce a novel approach for medical image segmentation that dynamically adjusts the intrinsic rank during adaptation. Viewing the low-rank representation of the trainable weight matrices as a singular value decomposition, we introduce an l_1 sparsity regularizer to the loss function, and tackle it with a proximal optimizer. The regularizer could be viewed as a penalty on the decomposition rank. Hence, its minimization enables to find task-adapted ranks automatically. Our method is evaluated in a realistic few-shot fine-tuning setting, where we compare it first to the standard LoRA and then to several other PEFT methods across two distinguishable tasks: base organs and novel organs. Our extensive experiments demonstrate the significant performance improvements driven by our method, highlighting its efficiency and robustness against suboptimal rank initialization. Our code is publicly available: https://github.com/ghassenbaklouti/ARENA