VERA-MH: Validation of Ethical and Responsible AI in Mental Health

TL;DR

This paper introduces VERA-MH, a clinically-validated evaluation framework for assessing the safety of chatbots in mental health support, focusing on suicidal ideation risks.

Contribution

It presents a novel multi-step evaluation method involving conversation simulation, clinical judgment, and model rating for mental health chatbot safety.

Findings

Evaluated four leading LLM providers using VERA-MH.

Developed clinically-guided user personas for simulation.

Structured rubric improves consistency in safety assessments.

Abstract

Chatbot usage has increased, including in fields for which they were never developed for--notably mental health support. To that end, we introduce Validations of Ethical and Responsible AI in Mental Health (VERA-MH), a novel clinically-validated evaluation for safety of chatbots in the context of mental health support. The first iteration of VERA-MH focuses on Suicidal Ideation (SI) risks, by assessing how well chatbots can responds to users that might be in crisis. VERA-MH is comprised of three steps: conversation simulation, conversation judging and model rating. First, to simulate conversations with the chatbot under evaluation, another chatbot is tasked with role-playing users based on specific personas. Such user personas have been developed under clinical guidance, to make sure that, among others, multiple risk factors, demographic characteristics and disclosure factors were represented. In the judging step, a second support model is used as an LLM-as-a-Judge, together with a clinically-developed rubric. The rubric is structured as a flow, with a single Yes/No question asked each time, to improve answers' consistency and highlight models' failure modes. In the last stage, results of each conversation are aggregated to present the final evaluation of the chatbot. Together with the framework, we present the result of the evaluations for four leading LLM providers.