Beyond Fine-Tuning: A Systematic Study of Sampling Techniques in Personalized Image Generation

TL;DR

This paper systematically analyzes sampling techniques in personalized image generation, proposing a decision framework to optimize concept fidelity, prompt adherence, and resource efficiency across various architectures.

Contribution

It introduces a comprehensive analysis of sampling strategies beyond fine-tuning and presents a decision framework for strategy selection in personalized image generation.

Findings

Sampling strategies significantly impact concept preservation and image quality.

The proposed framework effectively guides strategy choice based on user constraints.

Systematic analysis enables better understanding of sampling effects across architectures.

Abstract

Personalized text-to-image generation aims to create images tailored to user-defined concepts and textual descriptions. Balancing the fidelity of the learned concept with its ability for generation in various contexts presents a significant challenge. Existing methods often address this through diverse fine-tuning parameterizations and improved sampling strategies that integrate superclass trajectories during the diffusion process. While improved sampling offers a cost-effective, training-free solution for enhancing fine-tuned models, systematic analyses of these methods remain limited. Current approaches typically tie sampling strategies with fixed fine-tuning configurations, making it difficult to isolate their impact on generation outcomes. To address this issue, we systematically analyze sampling strategies beyond fine-tuning, exploring the impact of concept and superclass trajectories on the results. Building on this analysis, we propose a decision framework evaluating text alignment, computational constraints, and fidelity objectives to guide strategy selection. It integrates with diverse architectures and training approaches, systematically optimizing concept preservation, prompt adherence, and resource efficiency. The source code can be found at https://github.com/ControlGenAI/PersonGenSampler.