MultLFG: Training-free Multi-LoRA composition using Frequency-domain Guidance

TL;DR

MultLFG introduces a training-free, frequency-guided multi-LoRA fusion method that adaptively combines multiple adapters for improved compositional image synthesis without additional training.

Contribution

It proposes a novel frequency-domain guidance framework for adaptive, training-free multi-LoRA composition, enhancing visual coherence and control.

Findings

Outperforms state-of-the-art in compositional fidelity

Improves spatial coherence in multi-concept images

Enhances control over multi-LoRA fusion

Abstract

Low-Rank Adaptation (LoRA) has gained prominence as a computationally efficient method for fine-tuning generative models, enabling distinct visual concept synthesis with minimal overhead. However, current methods struggle to effectively merge multiple LoRA adapters without training, particularly in complex compositions involving diverse visual elements. We introduce MultLFG, a novel framework for training-free multi-LoRA composition that utilizes frequency-domain guidance to achieve adaptive fusion of multiple LoRAs. Unlike existing methods that uniformly aggregate concept-specific LoRAs, MultLFG employs a timestep and frequency subband adaptive fusion strategy, selectively activating relevant LoRAs based on content relevance at specific timesteps and frequency bands. This frequency-sensitive guidance not only improves spatial coherence but also provides finer control over multi-LoRA composition, leading to more accurate and consistent results. Experimental evaluations on the ComposLoRA benchmark reveal that MultLFG substantially enhances compositional fidelity and image quality across various styles and concept sets, outperforming state-of-the-art baselines in multi-concept generation tasks. Code will be released.