Say Cheese! Detail-Preserving Portrait Collection Generation via Natural Language Edits

TL;DR

This paper introduces a new task called Portrait Collection Generation (PCG) that creates coherent portrait collections from natural language edits, addressing complex multi-attribute modifications and detail preservation, and proposes a large dataset and a novel framework for this purpose.

Contribution

The paper presents the first large-scale PCG dataset and a novel framework, SCheese, for high-fidelity, detail-preserving portrait collection generation from natural language instructions.

Findings

CHEESE dataset contains 24K collections and 573K samples.

SCheese achieves state-of-the-art performance on PCG tasks.

Framework effectively preserves identity and details during generation.

Abstract

As social media platforms proliferate, users increasingly demand intuitive ways to create diverse, high-quality portrait collections. In this work, we introduce Portrait Collection Generation (PCG), a novel task that generates coherent portrait collections by editing a reference portrait image through natural language instructions. This task poses two unique challenges to existing methods: (1) complex multi-attribute modifications such as pose, spatial layout, and camera viewpoint; and (2) high-fidelity detail preservation including identity, clothing, and accessories. To address these challenges, we propose CHEESE, the first large-scale PCG dataset containing 24K portrait collections and 573K samples with high-quality modification text annotations, constructed through an Large Vison-Language Model-based pipeline with inversion-based verification. We further propose SCheese, a framework that combines text-guided generation with hierarchical identity and detail preservation. SCheese employs adaptive feature fusion mechanism to maintain identity consistency, and ConsistencyNet to inject fine-grained features for detail consistency. Comprehensive experiments validate the effectiveness of CHEESE in advancing PCG, with SCheese achieving state-of-the-art performance.