Visual Personalization Turing Test

TL;DR

The paper proposes the Visual Personalization Turing Test (VPTT), a new evaluation paradigm for visual personalization that assesses perceptual indistinguishability, supported by a comprehensive framework including benchmarks, models, and metrics.

Contribution

It introduces the VPTT framework, including a benchmark, a retrieval-augmented generator, and a calibrated score, advancing the evaluation of personalized visual content generation.

Findings

VPTT Score correlates well with human and VLM judgments.

VPRAG achieves a good balance between originality and alignment.

VPTT provides a scalable, privacy-safe method for evaluating visual personalization.

Abstract

We introduce the Visual Personalization Turing Test (VPTT), a new paradigm for evaluating contextual visual personalization based on perceptual indistinguishability, rather than identity replication. A model passes the VPTT if its output (image, video, 3D asset, etc.) is indistinguishable to a human or calibrated VLM judge from content a given person might plausibly create or share. To operationalize VPTT, we present the VPTT Framework, integrating a 10k-persona benchmark (VPTT-Bench), a visual retrieval-augmented generator (VPRAG), and the VPTT Score, a text-only metric calibrated against human and VLM judgments. We show high correlation across human, VLM, and VPTT evaluations, validating the VPTT Score as a reliable perceptual proxy. Experiments demonstrate that VPRAG achieves the best alignment-originality balance, offering a scalable and privacy-safe foundation for personalized generative AI.