Localizing Firearm Carriers by Identifying Human-Object Pairs

TL;DR

This paper introduces a novel method for identifying individuals carrying firearms in images by pairing detected humans with firearms and classifying these pairs, achieving high accuracy through spatial feature analysis.

Contribution

The paper presents a new approach using human-object pairing and classification, extending existing datasets, and demonstrating improved firearm localization accuracy.

Findings

Achieved $AP_{hold} = 78.5$ in firearm detection.

Effective use of human pose and hand key-points.

Enhanced dataset with human-firearm annotations.

Abstract

Visual identification of gunmen in a crowd is a challenging problem, that requires resolving the association of a person with an object (firearm). We present a novel approach to address this problem, by defining human-object interaction (and non-interaction) bounding boxes. In a given image, human and firearms are separately detected. Each detected human is paired with each detected firearm, allowing us to create a paired bounding box that contains both object and the human. A network is trained to classify these paired-bounding-boxes into human carrying the identified firearm or not. Extensive experiments were performed to evaluate effectiveness of the algorithm, including exploiting full pose of the human, hand key-points, and their association with the firearm. The knowledge of spatially localized features is key to success of our method by using multi-size proposals with adaptive average pooling. We have also extended a previously firearm detection dataset, by adding more images and tagging in extended dataset the human-firearm pairs (including bounding boxes for firearms and gunmen). The experimental results ($AP_{hold} = 78.5$) demonstrate effectiveness of the proposed method.