Multimodal learning enables instant ionizing radiation alerts on unmodified mobile phones for real-world emergency response

TL;DR

This paper introduces a novel multimodal deep learning approach enabling mobile phones to detect ionizing radiation instantly during emergencies without hardware modifications, significantly improving accessibility and response times.

Contribution

It presents the first practical mobile phone-based radiation detection method using multimodal deep learning without camera or hardware changes.

Findings

Detects hazardous radiation within six seconds with 86-96% accuracy.

Achieves 87% accuracy in low-level radiation detection over extended periods.

Operates effectively without additional hardware or camera coverage.

Abstract

In a radiation emergency, every second counts, yet the public rarely has immediate access to dedicated monitoring devices when they are needed most. Here, the first practical mobile phone-based emergency ionizing radiation detection method is presented that operates entirely without requiring camera coverage or additional hardware modifications. Utilizing a multimodal deep learning approach that integrates sparse radiation-induced signal distributions with the brightness patterns, the proposed framework effectively isolates subtle radiation signals from overwhelming visual interference. A hybrid 3D-2D convolutional neural network (CNN) identifies radiation-induced spots from raw mobile phone video, while a multi-layer perceptron (MLP) fuses the radiation signal and brightness maps for the dose rate estimation. The method detects hazardous dose rates (25-280 mRem/h) rapidly within six seconds (accuracy 86-96%), and low-level radiation (-0.6 mRem/h) with extended measurement durations achieves 87% accuracy. The developed method greatly enhances mobile phone radiation detection practicality and shows substantial potential as an accessible radiation emergency detection tool.