Finer-Personalization Rank: Fine-Grained Retrieval Examines Identity Preservation for Personalized Generation

TL;DR

This paper introduces Finer-Personalization Rank, a new evaluation method for personalized generative models that assesses identity preservation at multiple granularities using a ranking-based retrieval approach, revealing limitations of existing metrics.

Contribution

It proposes a gallery-based ranking evaluation protocol that better captures identity preservation in personalized generation, addressing shortcomings of semantic similarity metrics.

Findings

Finer-Personalization Rank more accurately reflects identity retention.

It reveals significant identity drift in popular personalization methods.

The protocol is effective across various benchmarks like CUB and Stanford Cars.

Abstract

The rise of personalized generative models raises a central question: how should we evaluate identity preservation? Given a reference image (e.g., one's pet), we expect the generated image to retain precise details attached to the subject's identity. However, current generative evaluation metrics emphasize the overall semantic similarity between the reference and the output, and overlook these fine-grained discriminative details. We introduce Finer-Personalization Rank, an evaluation protocol tailored to identity preservation. Instead of pairwise similarity, Finer-Personalization Rank adopts a ranking view: it treats each generated image as a query against an identity-labeled gallery consisting of visually similar real images. Retrieval metrics (e.g., mean average precision) measure performance, where higher scores indicate that identity-specific details (e.g., a distinctive head spot) are preserved. We assess identity at multiple granularities -- from fine-grained categories (e.g., bird species, car models) to individual instances (e.g., re-identification). Across CUB, Stanford Cars, and animal Re-ID benchmarks, Finer-Personalization Rank more faithfully reflects identity retention than semantic-only metrics and reveals substantial identity drift in several popular personalization methods. These results position the gallery-based protocol as a principled and practical evaluation for personalized generation.