METEOR:A Dense, Heterogeneous, and Unstructured Traffic Dataset With Rare Behaviors

TL;DR

METEOR is a comprehensive traffic dataset capturing rare multi-agent driving behaviors in unstructured scenarios, challenging current perception models and fostering development of more advanced methods.

Contribution

The paper introduces METEOR, a large-scale, diverse traffic dataset with annotations for rare behaviors, enabling benchmarking and development of perception models in complex scenarios.

Findings

State-of-the-art models fail on METEOR, unlike on existing datasets.

METEOR captures diverse, rare traffic behaviors and conditions.

Benchmarking reveals gaps in current perception methods.

Abstract

We present a new traffic dataset, METEOR, which captures traffic patterns and multi-agent driving behaviors in unstructured scenarios. METEOR consists of more than 1000 one-minute videos, over 2 million annotated frames with bounding boxes and GPS trajectories for 16 unique agent categories, and more than 13 million bounding boxes for traffic agents. METEOR is a dataset for rare and interesting, multi-agent driving behaviors that are grouped into traffic violations, atypical interactions, and diverse scenarios. Every video in METEOR is tagged using a diverse range of factors corresponding to weather, time of the day, road conditions, and traffic density. We use METEOR to benchmark perception methods for object detection and multi-agent behavior prediction. Our key finding is that state-of-the-art models for object detection and behavior prediction, which otherwise succeed on existing datasets such as Waymo, fail on the METEOR dataset. METEOR marks the first step towards the development of more sophisticated perception models for dense, heterogeneous, and unstructured scenarios.