Adaptive Subspace Projection for Generative Personalization

TL;DR

This paper introduces AdaptSP, a training-free method that uses adaptive subspace projection to mitigate semantic collapsing in generative personalization, improving prompt fidelity and contextual relevance.

Contribution

It identifies the low-dimensional subspace responsible for semantic drift and proposes AdaptSP to correct this drift without additional training.

Findings

AdaptSP effectively reduces semantic collapsing in experiments.

The method improves the alignment of generated content with prompts.

AdaptSP maintains subject identity while enhancing contextual fidelity.

Abstract

Generative personalization often suffers from the semantic collapsing problem (SCP), where a learned personalized concept overpowers the rest of the text prompt, causing the model to ignore important contextual details. To address this, we first analyze the underlying cause, revealing that the semantic drift responsible for SCP is not random but is concentrated within a specific low-dimensional subspace. We also discover that the personalization process perturbs the embedding of the original base concept, making it an unstable reference point. Based on these insights, we introduce Test-time Embedding Adjustment with Adaptive Subspace Projection (AdaptSP), a training-free method that uses the stable, pre-trained embedding as an anchor. AdaptSP isolates the semantic drift and projects it onto the identified subspace, performing a precise adjustment that mitigates SCP while maintaining the subject identity. Our experiments show that this targeted approach significantly improves prompt fidelity and contextual alignment.