A Survey on Personalized Content Synthesis with Diffusion Models

TL;DR

This survey comprehensively reviews recent advancements in Personalized Content Synthesis using diffusion models, categorizing methods into test-time fine-tuning and pre-trained adaptation, and discusses challenges and future directions.

Contribution

It provides the first up-to-date, detailed overview of PCS techniques beyond text-to-image, analyzing methodologies, tasks, challenges, and proposing future research directions.

Findings

Over 150 methods in PCS have been introduced in the past two years.

PCS techniques are mainly categorized into TTF and PTA approaches.

Key challenges include overfitting and balancing subject fidelity with text alignment.

Abstract

Recent advancements in diffusion models have significantly impacted content creation, leading to the emergence of Personalized Content Synthesis (PCS). By utilizing a small set of user-provided examples featuring the same subject, PCS aims to tailor this subject to specific user-defined prompts. Over the past two years, more than 150 methods have been introduced in this area. However, existing surveys primarily focus on text-to-image generation, with few providing up-to-date summaries on PCS. This paper provides a comprehensive survey of PCS, introducing the general frameworks of PCS research, which can be categorized into test-time fine-tuning (TTF) and pre-trained adaptation (PTA) approaches. We analyze the strengths, limitations, and key techniques of these methodologies. Additionally, we explore specialized tasks within the field, such as object, face, and style personalization, while highlighting their unique challenges and innovations. Despite the promising progress, we also discuss ongoing challenges, including overfitting and the trade-off between subject fidelity and text alignment. Through this detailed overview and analysis, we propose future directions to further the development of PCS.