MOTOR: A Multimodal Dataset for Two-Wheeler Rider Behavior Understanding

TL;DR

The MOTOR dataset is a comprehensive multimodal resource for analyzing two-wheeler rider behavior, aiming to improve safety and assist in developing advanced driver assistance systems for dense traffic environments.

Contribution

It introduces the first large-scale, multimodal dataset specifically for two-wheeler rider behavior in dense traffic, enabling new research opportunities.

Findings

Multimodal fusion of RGB, gaze, and telemetry improves behavior recognition.

Benchmark results show state-of-the-art models benefit from multimodal data.

Dataset and code are publicly available for research use.

Abstract

Two-wheelers account for a disproportionately high share of road fatalities in the Global South. Research on two-wheeler rider behavior, however, lags far behind four-wheelers, where multimodal datasets have driven major advances in Advanced Driver Assistance Systems (ADAS). To address this gap, we present the MOtorized TwO-wheeler Rider (MOTOR) dataset, the first large-scale, multi-view, multimodal resource dedicated to two-wheelers in dense, unstructured traffic. MOTOR comprises 1,629 sequences (25+ hours of video data) collected from 16 riders and integrates synchronized front, rear, and helmet videos, rider eye-gaze from wearable trackers, on-road audio, and telemetry (GPS, accelerometer, gyroscope). Rich annotations capture traffic context, rider state, 12 riding maneuvers spanning conventional and unconventional behaviors, and legality labels (Legal, Illegal, Unspecified). We benchmark rider behavior recognition and maneuver legality classification using state-of-the-art video action recognition backbones (CNN and Transformer-based), extended with multimodal fusion, and find that combining RGB, gaze, and telemetry consistently yields the best performance. MOTOR thus provides a unique foundation for advancing safety-critical understanding of two-wheeler riding. It offers the research community a benchmark to develop and evaluate models for behavior analysis, legality-aware prediction, and intelligent transportation systems. Dataset and code is available at https: //varuniiith.github.io/MOTOR-Dataset/