GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety

TL;DR

GrandGuard introduces a comprehensive framework with a taxonomy, benchmark, and safeguards to address elderly-specific risks in LLM-based chatbots, aiming to improve safety for aging users.

Contribution

It provides the first detailed taxonomy and benchmark for elderly-specific risks and demonstrates effective safeguards to mitigate these risks in LLM interactions.

Findings

Leading LLMs mishandle elderly risks in over 50% of cases.

Fine-tuned Llama-Guard-3 detects unsafe prompts with 96.2% accuracy.

Policy-enhanced safeguards achieve 90.9% detection accuracy.

Abstract

As older adults increasingly use LLM-based chatbots for companionship and assistance, a safety gap is emerging. Older adults may face vulnerabilities from social isolation, limited digital literacy, and cognitive decline, yet existing safety benchmarks largely target general harms and overlook elderly-specific risks. For example, a prompt such as "how to repair a ceiling light alone in the dark" may be benign for most users but poses a serious fall risk for older adults with mobility limitations. We introduce GrandGuard, the first comprehensive framework for assessing and mitigating elderly-specific contextual risks in LLM interactions. We develop a three-level taxonomy with 50 fine-grained risk types across mental well-being, financial, medical, toxicity, and privacy domains, grounded in real-world incidents, community discussions, and analysis of stakeholder studies. Using this taxonomy, we construct a benchmark of 10,404 labeled prompts and responses, showing that several leading LLMs mishandle elderly-specific contextual risks in over 50% of cases. We mitigate these failures with two safeguards: a fine-tuned Llama-Guard-3 and a policy-enhanced gpt-oss-safeguard-20b, achieving up to 96.2% and 90.9% unsafe-prompt detection accuracy, respectively. GrandGuard lays the groundwork for AI systems that move beyond general safety to support aging populations.