AI-based Pilgrim Detection using Convolutional Neural Networks

TL;DR

This paper presents an AI-based system using convolutional neural networks to detect and identify pilgrims in large-scale surveillance footage, enhancing safety monitoring during the Hajj pilgrimage.

Contribution

It introduces a new dataset for pilgrim detection and compares YOLOv3 and Faster R-CNN models, demonstrating improved accuracy with Faster R-CNN for this application.

Findings

Faster R-CNN achieved 51% mean average precision.

A comprehensive dataset for pilgrim detection was created.

Deep learning models effectively identify pilgrims and their features.

Abstract

Pilgrimage represents the most important Islamic religious gathering in the world where millions of pilgrims visit the holy places of Makkah and Madinah to perform their rituals. The safety and security of pilgrims is the highest priority for the authorities. In Makkah, 5000 cameras are spread around the holy for monitoring pilgrims, but it is almost impossible to track all events by humans considering the huge number of images collected every second. To address this issue, we propose to use artificial intelligence technique based on deep learning and convolution neural networks to detect and identify Pilgrims and their features. For this purpose, we built a comprehensive dataset for the detection of pilgrims and their genders. Then, we develop two convolutional neural networks based on YOLOv3 and Faster-RCNN for the detection of Pilgrims. Experiments results show that Faster RCNN with Inception v2 feature extractor provides the best mean average precision over all classes of 51%.