Learning to Assess Danger from Movies for Cooperative Escape Planning in Hazardous Environments
Vikram Shree, Sarah Allen, Beatriz Asfora, Jacopo Banfi, Mark Campbell

TL;DR
This paper introduces a novel multi-modal perception framework using movie data to assess danger in hazardous environments, enhancing robot perception and planning for safer human-robot escape missions.
Contribution
It presents a new dataset derived from movies for danger assessment and a Bayesian multi-modal pipeline integrating visual and language data for improved hazard estimation.
Findings
Higher success rate in simulated rescue missions
Effective fusion of visual and language data for danger estimation
Risk-aware planning improves safety in escape routes
Abstract
There has been a plethora of work towards improving robot perception and navigation, yet their application in hazardous environments, like during a fire or an earthquake, is still at a nascent stage. We hypothesize two key challenges here: first, it is difficult to replicate such scenarios in the real world, which is necessary for training and testing purposes. Second, current systems are not fully able to take advantage of the rich multi-modal data available in such hazardous environments. To address the first challenge, we propose to harness the enormous amount of visual content available in the form of movies and TV shows, and develop a dataset that can represent hazardous environments encountered in the real world. The data is annotated with high-level danger ratings for realistic disaster images, and corresponding keywords are provided that summarize the content of the scene. In…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Robotic Path Planning Algorithms · Robotics and Sensor-Based Localization
