PROMPT2BOX: Uncovering Entailment Structure among LLM Prompts

TL;DR

PROMPT2BOX introduces a novel box embedding method for prompts that captures both semantic similarity and specificity, improving the analysis of LLM weaknesses and hierarchical prompt structure.

Contribution

It proposes a new box embedding space and a dimension reduction technique to better represent prompt specificity and facilitate visualization, outperforming vector baselines.

Findings

Box embeddings better capture prompt specificity than vector embeddings.

PROMPT2BOX identifies 8.9% more LLM weaknesses.

Hierarchical depth correlates 33% more with instruction specificity.

Abstract

To discover the weaknesses of LLMs, researchers often embed prompts into a vector space and cluster them to extract insightful patterns. However, vector embeddings primarily capture topical similarity. As a result, prompts that share a topic but differ in specificity, and consequently in difficulty, are often represented similarly, making fine-grained weakness analysis difficult. To address this limitation, we propose PROMPT2BOX, which embeds prompts into a box embedding space using a trained encoder. The encoder, trained on existing and synthesized datasets, outputs box embeddings that capture not only semantic similarity but also specificity relations between prompts (e.g., "writing an adventure story" is more specific than "writing a story"). We further develop a novel dimension reduction technique for box embeddings to facilitate dataset visualization and comparison. Our experiments demonstrate that box embeddings consistently capture prompt specificity better than vector baselines. On the downstream task of creating hierarchical clustering trees for 17 LLMs from the UltraFeedback dataset, PROMPT2BOX can identify 8.9\% more LLM weaknesses than vector baselines and achieves an approximately 33\% stronger correlation between hierarchical depth and instruction specificity.