Application-Driven AI Paradigm for Hand-Held Action Detection

TL;DR

This paper introduces an application-driven hierarchical AI framework for hand-held action detection, improving accuracy in safety-critical scenarios by focusing on detailed hand and object features.

Contribution

It proposes a novel coarse-to-fine hierarchical detection approach tailored for hand-held actions, enhancing detection accuracy and robustness in complex environments.

Findings

Higher detection rate achieved in real-world scenarios

Good adaptation and robustness demonstrated

Effective for safety-critical applications

Abstract

In practical applications especially with safety requirement, some hand-held actions need to be monitored closely, including smoking cigarettes, dialing, eating, etc. Taking smoking cigarettes as example, existing smoke detection algorithms usually detect the cigarette or cigarette with hand as the target object only, which leads to low accuracy. In this paper, we propose an application-driven AI paradigm for hand-held action detection based on hierarchical object detection. It is a coarse-to-fine hierarchical detection framework composed of two modules. The first one is a coarse detection module with the human pose consisting of the whole hand, cigarette and head as target object. The followed second one is a fine detection module with the fingers holding cigarette, mouth area and the whole cigarette as target. Some experiments are done with the dataset collected from real-world scenarios, and the results show that the proposed framework achieve higher detection rate with good adaptation and robustness in complex environments.