(Safe) SMART Hands: Hand Activity Analysis and Distraction Alerts Using a Multi-Camera Framework

TL;DR

This paper presents SMART Hands, a multi-camera machine learning framework that accurately classifies driver hand activity, enhancing safety by monitoring hand positions and distractions in real-time.

Contribution

Introduces a novel multi-camera ensemble framework for driver hand activity classification with high accuracy, improving passive driver monitoring systems.

Findings

Achieved 98% classification accuracy across various hand positions and objects.

Demonstrated effectiveness of multi-camera setup for driver hand activity analysis.

Framework can be extended to gaze and pose analysis for safety applications.

Abstract

Manual (hand-related) activity is a significant source of crash risk while driving. Accordingly, analysis of hand position and hand activity occupation is a useful component to understanding a driver's readiness to take control of a vehicle. Visual sensing through cameras provides a passive means of observing the hands, but its effectiveness varies depending on camera location. We introduce an algorithmic framework, SMART Hands, for accurate hand classification with an ensemble of camera views using machine learning. We illustrate the effectiveness of this framework in a 4-camera setup, reaching 98% classification accuracy on a variety of locations and held objects for both of the driver's hands. We conclude that this multi-camera framework can be extended to additional tasks such as gaze and pose analysis, with further applications in driver and passenger safety.