It's all about you: Personalized in-Vehicle Gesture Recognition with a Time-of-Flight Camera

TL;DR

This paper presents a personalized in-vehicle gesture recognition system using a Time-of-Flight camera, combining hardware and algorithmic enhancements to improve accuracy and user adaptation in driving environments.

Contribution

It introduces a model-adaptation approach for CNN-LSTM models that personalizes gesture recognition, reducing data needs and increasing accuracy in dynamic driving conditions.

Findings

Achieved up to 90% recognition accuracy.

Personalized adaptation improves recognition performance.

Incremental learning enhances system robustness.

Abstract

Despite significant advances in gesture recognition technology, recognizing gestures in a driving environment remains challenging due to limited and costly data and its dynamic, ever-changing nature. In this work, we propose a model-adaptation approach to personalize the training of a CNNLSTM model and improve recognition accuracy while reducing data requirements. Our approach contributes to the field of dynamic hand gesture recognition while driving by providing a more efficient and accurate method that can be customized for individual users, ultimately enhancing the safety and convenience of in-vehicle interactions, as well as driver's experience and system trust. We incorporate hardware enhancement using a time-of-flight camera and algorithmic enhancement through data augmentation, personalized adaptation, and incremental learning techniques. We evaluate the performance of our approach in terms of recognition accuracy, achieving up to 90\%, and show the effectiveness of personalized adaptation and incremental learning for a user-centered design.