Real-World Design and Deployment of an Embedded GenAI-powered 9-1-1 Calltaking Training System: Experiences and Lessons Learned

TL;DR

This paper reports on the real-world design, deployment, and lessons learned from implementing a GenAI-powered 9-1-1 call-taking training system in Nashville, highlighting practical challenges and solutions in a safety-critical setting.

Contribution

It presents the first large-scale deployment of an embedded GenAI training system in a public safety environment, offering practical insights and governance practices.

Findings

Deployment scaled to 190 users over 1,120 sessions

Identified challenges in system delivery, resilience, and human factors

Provided grounded guidance for AI integration in safety-critical contexts

Abstract

Emergency call-takers form the first operational link in public safety response, handling over 240 million calls annually while facing a sustained training crisis: staffing shortages exceed 25\% in many centers, and preparing a single new hire can require up to 720 hours of one-on-one instruction that removes experienced personnel from active duty. Traditional training approaches struggle to scale under these constraints, limiting both coverage and feedback timeliness. In partnership with Metro Nashville Department of Emergency Communications (MNDEC), we designed, developed, and deployed a GenAI-powered call-taking training system under real-world constraints. Over six months, deployment scaled from initial pilot to 190 operational users across 1,120 training sessions, exposing systematic challenges around system delivery, rigor, resilience, and human factors that remain largely invisible in controlled or purely simulated evaluations. By analyzing deployment logs capturing 98,429 user interactions, organizational processes, and stakeholder engagement patterns, we distill four key lessons, each coupled with concrete design and governance practices. These lessons provide grounded guidance for researchers and practitioners seeking to embed AI-driven training systems in safety-critical public sector environments where embedded constraints fundamentally shape socio-technical design.