Concentration bounds on response-based vector embeddings of black-box generative models

TL;DR

This paper derives concentration bounds for response-based vector embeddings of black-box generative models, providing theoretical guarantees on the number of samples needed for accurate population-level embedding approximation.

Contribution

It establishes high probability concentration bounds for Data Kernel Perspective Space embeddings under regularity conditions, advancing the theoretical understanding of embeddings for generative models.

Findings

Provides sample complexity bounds for embedding accuracy

Uses algebraic tools applicable to noisy multidimensional scaling

Enhances statistical analysis of black-box generative models

Abstract

Generative models, such as large language models or text-to-image diffusion models, can generate relevant responses to user-given queries. Response-based vector embeddings of generative models facilitate statistical analysis and inference on a given collection of black-box generative models. The Data Kernel Perspective Space embedding is one particular method of obtaining response-based vector embeddings for a given set of generative models, already discussed in the literature. In this paper, under appropriate regularity conditions, we establish high probability concentration bounds on the sample vector embeddings for a given set of generative models, obtained through the method of Data Kernel Perspective Space embedding. Our results tell us the required number of sample responses needed in order to approximate the population-level vector embeddings with a desired level of accuracy. The algebraic tools used to establish our results can be used further for establishing concentration bounds on Classical Multidimensional Scaling embeddings in general, when the dissimilarities are observed with noise.