Parameter-Efficient Fine-Tuning of 3D DDPM for MRI Image Generation Using Tensor Networks

TL;DR

This paper introduces TenVOO, a tensor network-based parameter-efficient fine-tuning method for 3D MRI image generation with DDPMs, achieving state-of-the-art results with minimal trainable parameters.

Contribution

We propose TenVOO, a novel tensor network approach for efficient fine-tuning of 3D diffusion models in MRI, significantly reducing parameters while maintaining high performance.

Findings

Outperforms existing methods in MS-SSIM on MRI datasets

Uses only 0.3% of original model parameters for fine-tuning

Achieves state-of-the-art spatial dependency modeling in 3D MRI generation

Abstract

We address the challenge of parameter-efficient fine-tuning (PEFT) for three-dimensional (3D) U-Net-based denoising diffusion probabilistic models (DDPMs) in magnetic resonance imaging (MRI) image generation. Despite its practical significance, research on parameter-efficient representations of 3D convolution operations remains limited. To bridge this gap, we propose Tensor Volumetric Operator (TenVOO), a novel PEFT method specifically designed for fine-tuning DDPMs with 3D convolutional backbones. Leveraging tensor network modeling, TenVOO represents 3D convolution kernels with lower-dimensional tensors, effectively capturing complex spatial dependencies during fine-tuning with few parameters. We evaluate TenVOO on three downstream brain MRI datasets-ADNI, PPMI, and BraTS2021-by fine-tuning a DDPM pretrained on 59,830 T1-weighted brain MRI scans from the UK Biobank. Our results demonstrate that TenVOO achieves state-of-the-art performance in multi-scale structural similarity index measure (MS-SSIM), outperforming existing approaches in capturing spatial dependencies while requiring only 0.3% of the trainable parameters of the original model. Our code is available at: https://github.com/xiaovhua/tenvoo