CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptions

TL;DR

This paper introduces the CityFlow-NL benchmark, a novel dataset with natural language descriptions for vehicle tracking and retrieval at city scale, enabling research on multi-object tracking and event localization using language cues.

Contribution

It extends the CityFlow Benchmark with over 5,000 NL descriptions, creating the first dataset for multi-target multi-camera vehicle tracking and retrieval by natural language.

Findings

First dataset for vehicle tracking and retrieval with NL descriptions

Facilitates research on multi-object tracking and event localization using language

Provides tasks and benchmarks for future development in NL-based vehicle tracking

Abstract

Natural Language (NL) descriptions can be one of the most convenient or the only way to interact with systems built to understand and detect city scale traffic patterns and vehicle-related events. In this paper, we extend the widely adopted CityFlow Benchmark with NL descriptions for vehicle targets and introduce the CityFlow-NL Benchmark. The CityFlow-NL contains more than 5,000 unique and precise NL descriptions of vehicle targets, making it the first multi-target multi-camera tracking with NL descriptions dataset to our knowledge. Moreover, the dataset facilitates research at the intersection of multi-object tracking, retrieval by NL descriptions, and temporal localization of events. In this paper, we focus on two foundational tasks: the Vehicle Retrieval by NL task and the Vehicle Tracking by NL task, which take advantage of the proposed CityFlow-NL benchmark and provide a strong basis for future research on the multi-target multi-camera tracking by NL description task.