WaRA: Wavelet Low Rank Adaptation

TL;DR

WaRA introduces a wavelet-based low-rank adaptation method for fine-tuning large pretrained vision models on medical images, effectively capturing multi-scale features with high efficiency.

Contribution

The paper proposes WaRA, a novel wavelet-structured low-rank adaptation module for efficient medical image classification, and Tiny-WaRA, a parameter-reduction variant for extremely low-resource scenarios.

Findings

WaRA outperforms strong PEFT baselines across four medical imaging modalities.

WaRA maintains high efficiency with a compact trainable interface.

Tiny-WaRA achieves competitive performance with minimal trainable parameters.

Abstract

Adapting large pretrained vision models to medical image classification is often limited by memory, computation, and task-specific specializations. Parameter-efficient fine-tuning (PEFT) methods like LoRA reduce this cost by learning low-rank updates, but operating directly in feature space can struggle to capture the localized, multi-scale features common in medical imaging. We propose WaRA, a wavelet-structured adaptation module that performs low-rank adaptation in a wavelet domain. WaRA reshapes patch tokens into a spatial grid, applies a fixed discrete wavelet transform, updates subband coefficients using a shared low-rank adapter, and reconstructs the additive update through an inverse wavelet transform. This design provides a compact trainable interface while biasing the update toward both coarse structure and fine detail. For extremely low-resource settings, we introduce Tiny-WaRA, which further reduces trainable parameters by learning only a small set of coefficients in a fixed basis derived from the pretrained weights through a truncated SVD. Experiments on medical image classification across four modalities and datasets demonstrate that WaRA consistently improves performance over strong PEFT baselines, while retaining a favorable efficiency profile. Our code is publicly available at~\href{https://github.com/moeinheidari7829/WaRA}{\textcolor{magenta}{GitHub}}.