Designing Multi-Robot Ground Video Sensemaking with Public Safety Professionals

TL;DR

This paper introduces a new multi-robot ground video sensemaking testbed and a tool that uses large language models to enhance situational awareness for public safety, reducing workload and increasing confidence.

Contribution

It presents the first testbed for multi-robot ground video sensemaking and a novel tool integrating LLMs to improve public safety workflows.

Findings

Participants experienced reduced manual workload.

Participants gained greater confidence with LLM explanations.

Concerns about false alarms and privacy were noted.

Abstract

Videos from fleets of ground robots can advance public safety by providing scalable situational awareness and reducing professionals' burden. Yet little is known about how to design and integrate multi-robot videos into public safety workflows. Collaborating with six police agencies, we examined how such videos could be made practical. In Study 1, we presented the first testbed for multi-robot ground video sensemaking. The testbed includes 38 events-of-interest (EoI) relevant to public safety, a dataset of 20 robot patrol videos (10 day/night pairs) covering EoI types, and 6 design requirements aimed at improving current video sensemaking practices. In Study 2, we built MRVS, a tool that augments multi-robot patrol video streams with a prompt-engineered video understanding model. Participants reported reduced manual workload and greater confidence with LLM-based explanations, while noting concerns about false alarms and privacy. We conclude with implications for designing future multi-robot video sensemaking tools.