Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent Approach

TL;DR

This paper introduces a personalized safety benchmark and a planning-based agent for LLMs, demonstrating that personalized user information significantly enhances safety and proposing a practical, low-cost method for safe, user-specific responses.

Contribution

It presents PENGUIN, a comprehensive benchmark for personalized safety, and RAISE, a novel, training-free agent framework that improves safety by selectively acquiring user background information.

Findings

Personalized safety scores improve by 43.2% with user info.

RAISE enhances safety scores by up to 31.6%.

Low interaction cost of 2.7 queries per user.

Abstract

Large language models (LLMs) typically generate identical or similar responses for all users given the same prompt, posing serious safety risks in high-stakes applications where user vulnerabilities differ widely. Existing safety evaluations primarily rely on context-independent metrics - such as factuality, bias, or toxicity - overlooking the fact that the same response may carry divergent risks depending on the user's background or condition. We introduce personalized safety to fill this gap and present PENGUIN - a benchmark comprising 14,000 scenarios across seven sensitive domains with both context-rich and context-free variants. Evaluating six leading LLMs, we demonstrate that personalized user information significantly improves safety scores by 43.2%, confirming the effectiveness of personalization in safety alignment. However, not all context attributes contribute equally to safety enhancement. To address this, we develop RAISE - a training-free, two-stage agent framework that strategically acquires user-specific background. RAISE improves safety scores by up to 31.6% over six vanilla LLMs, while maintaining a low interaction cost of just 2.7 user queries on average. Our findings highlight the importance of selective information gathering in safety-critical domains and offer a practical solution for personalizing LLM responses without model retraining. This work establishes a foundation for safety research that adapts to individual user contexts rather than assuming a universal harm standard.