(Safe) SMART Hands: Hand Activity Analysis and Distraction Alerts Using a Multi-Camera Framework
Ross Greer, Lulua Rakla, Anish Gopalan, Mohan Trivedi

TL;DR
This paper presents SMART Hands, a multi-camera machine learning framework that accurately classifies driver hand activity, enhancing safety by monitoring hand positions and distractions in real-time.
Contribution
Introduces a novel multi-camera ensemble framework for driver hand activity classification with high accuracy, improving passive driver monitoring systems.
Findings
Achieved 98% classification accuracy across various hand positions and objects.
Demonstrated effectiveness of multi-camera setup for driver hand activity analysis.
Framework can be extended to gaze and pose analysis for safety applications.
Abstract
Manual (hand-related) activity is a significant source of crash risk while driving. Accordingly, analysis of hand position and hand activity occupation is a useful component to understanding a driver's readiness to take control of a vehicle. Visual sensing through cameras provides a passive means of observing the hands, but its effectiveness varies depending on camera location. We introduce an algorithmic framework, SMART Hands, for accurate hand classification with an ensemble of camera views using machine learning. We illustrate the effectiveness of this framework in a 4-camera setup, reaching 98% classification accuracy on a variety of locations and held objects for both of the driver's hands. We conclude that this multi-camera framework can be extended to additional tasks such as gaze and pose analysis, with further applications in driver and passenger safety.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman-Automation Interaction and Safety · Gaze Tracking and Assistive Technology · Ergonomics and Musculoskeletal Disorders
