Hybrid Diffusion Model for Breast Ultrasound Image Augmentation

TL;DR

This paper introduces a hybrid diffusion framework that enhances breast ultrasound image augmentation by combining text-to-image, image refinement, and low-rank adaptation, resulting in more realistic synthetic images.

Contribution

The proposed method improves ultrasound image augmentation by integrating multiple diffusion techniques, achieving higher fidelity and class consistency compared to standard models.

Findings

Reduced FID from 45.97 to 33.29 compared to baseline

Generated realistic, class-consistent ultrasound images

Enhanced augmentation quality for diagnostic modeling

Abstract

We propose a hybrid diffusion-based augmentation framework to overcome the critical challenge of ultrasound data augmentation in breast ultrasound (BUS) datasets. Unlike conventional diffusion-based augmentations, our approach improves visual fidelity and preserves ultrasound texture by combining text-to-image generation with image-to-image (img2img) refinement, as well as fine-tuning with low-rank adaptation (LoRA) and textual inversion (TI). Our method generated realistic, class-consistent images on an open-source Kaggle breast ultrasound image dataset (BUSI). Compared to the Stable Diffusion v1.5 baseline, incorporating TI and img2img refinement reduced the Frechet Inception Distance (FID) from 45.97 to 33.29, demonstrating a substantial gain in fidelity while maintaining comparable downstream classification performance. Overall, the proposed framework effectively mitigates the low-fidelity limitations of synthetic ultrasound images and enhances the quality of augmentation for robust diagnostic modeling.