WatchPed: Pedestrian Crossing Intention Prediction Using Embedded Sensors of Smartwatch

TL;DR

This paper introduces a novel pedestrian crossing intention prediction model that combines smartwatch motion sensor data with visual input, improving accuracy especially in challenging visual conditions and large distances.

Contribution

It presents the first integrated model using smartwatch sensors for pedestrian intention prediction and introduces a new dataset with synchronized sensor and video data.

Findings

Model outperforms state-of-the-art in low-visibility conditions.

Enhanced accuracy at distances over 70 meters.

Introduces a new dataset with 255 synchronized video and sensor clips.

Abstract

The pedestrian crossing intention prediction problem is to estimate whether or not the target pedestrian will cross the street. State-of-the-art techniques heavily depend on visual data acquired through the front camera of the ego-vehicle to make a prediction of the pedestrian's crossing intention. Hence, the efficiency of current methodologies tends to decrease notably in situations where visual input is imprecise, for instance, when the distance between the pedestrian and ego-vehicle is considerable or the illumination levels are inadequate. To address the limitation, in this paper, we present the design, implementation, and evaluation of the first-of-its-kind pedestrian crossing intention prediction model based on integration of motion sensor data gathered through the smartwatch (or smartphone) of the pedestrian. We propose an innovative machine learning framework that effectively integrates motion sensor data with visual input to enhance the predictive accuracy significantly, particularly in scenarios where visual data may be unreliable. Moreover, we perform an extensive data collection process and introduce the first pedestrian intention prediction dataset that features synchronized motion sensor data. The dataset comprises 255 video clips that encompass diverse distances and lighting conditions. We trained our model using the widely-used JAAD and our own datasets and compare the performance with a state-of-the-art model. The results demonstrate that our model outperforms the current state-of-the-art method, particularly in cases where the distance between the pedestrian and the observer is considerable (more than 70 meters) and the lighting conditions are inadequate.